ISSUES& ANSWERS 



REL 2010-No. 087 




The relationship 
between changes in 
the percentage of 
students passing and 
in the percentage 
testing advanced 
on state assessment 
tests in Kentucky 
and Virginia 








NATIONAL CENTER for 

EDUCATION EVALUATION 
and REGIONAL ASSISTANCE 



Institute of Education Sciences 





REL 2010-No. 087 



ISSUES (^ANSWERS 




APPALACHIA 



Regional Educational Laboratory 
AtCNA 



The relationship between changes in the 
percentage of students passing and in 
the percentage testing advanced on state 
assessment tests in Kentucky and Virginia 



March 2010 



Prepared by 

Louis Jacobson 
CNA Education 

Laura Holian 
CNA Education 




NATIONAL CENTER for 

EDUCATION EVALUATION 
and REGIONAL ASSISTANCE 



Institute of Education Sciences 




Issues & Answers is an ongoing series of reports from short-term Fast Response Projects conducted by the regional educa- 
tional laboratories on current education issues of importance at local, state, and regional levels. Fast Response Project topics 
change to reflect new issues, as identified through lab outreach and requests for assistance from policymakers and educa- 
tors at state and local levels and from communities, businesses, parents, families, and youth. All Issues & Answers reports 
meet Institute of Education Sciences standards for scientifically valid research. 

March 2010 

This report was prepared for the Institute of Education Sciences (IES) under Contract ED-06-CO-0021 by Regional Educa- 
tional Laboratory Appalachia administered by CNA Education. The content of the publication does not necessarily reflect 
the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or 
organizations imply endorsement by the U.S. Government. 

This report is in the public domain. While permission to reprint this publication is not necessary, it should be cited as: 

Jacobson, L., and Flolian, L. (2010). The relationship between changes in the percentage of students passing and in the per- 
centage testing advanced on state assessment tests in Kentucky and Virginia (Issues & Answers Report, REL 2010-No. 087). 
Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation 
and Regional Assistance, Regional Educational Laboratory Appalachia. Retrieved from http://ies.ed.gov/ncee/edlabs. 

This report is available on the regional educational laboratory web site at http://ies.ed.gov/ncee/edlabs. 



Summary REL 2010-No. 087 

The relationship between changes in 
the percentage of students passing 
and in the percentage testing 
advanced on state assessment 
tests in Kentucky and Virginia 



The No Child Left Behind Act of 2001 
requires states to test students in read- 
ing and math and identify them as below 
proficient, proficient, or advanced. 
Schools are held accountable only for 
ensuring that students test proficient or 
better (both considered passing), leading 
to concerns that a focus on increasing the 
percentage of students testing proficient 
on state assessments may have an unin- 
tended consequence of reducing — or not 
increasing — the percentage of students 
testing advanced. Analysis of the data in 
Kentucky and Virginia finds that schools 
with the greatest increases in the per- 
centage passing also have the great- 
est increases in the percentage testing 
advanced. 

The No Child Left Behind (NCLB) Act of 2001 
requires states to test all students in reading 
and math and identify them as below profi- 
cient, proficient, or advanced, but the law holds 
schools accountable only for ensuring that 
students test proficient or better. State officials, 
district and school leaders, and others have 
expressed concern that attention to proficiency 
alone might have unintended consequences 



if focusing on moving students from below 
proficient to proficient has negative conse- 
quences on moving students from proficient to 
advanced. An alternative view sees attention to 
below proficient students as possibly improving 
the performance of all students. Few empiri- 
cal studies have examined how changes from 
proficient to advanced are affected by changes 
from below proficient to proficient. 

This study examines the statistical association 
between changes in the percentage of students 
in a school testing below proficient, proficient, 
and advanced in Kentucky and Virginia in the 
early years of NCLB accountability. The study 
was designed to answer four questions: 

1. What are the overall school-level trends in 
the percentage of students passing (testing 
proficient or advanced) and the percentage 
testing advanced on state assessment tests 
in Kentucky and Virginia? 

2. What is the statistical association between 
annualized changes in the school-level 
percentage of students passing and annu- 
alized changes in the school-level percent- 
age testing advanced? 



SUMMARY 



3. Does this association vary when control- 
ling for the percentage of students passing 
in 2001/02? 

4. Does this association vary when con- 
trolling for characteristics of a school’s 
students, such as level and change in the 
percentage of students eligible for free or 
reduced-price lunch, level and change in 
the percentage of racial/ethnic minority 
students, and the school’s locale (urban, 
suburban, town, or rural)? 

The study answered these questions using data 
on the percentages of students testing below 
proficient, proficient, and advanced provided 
by the Kentucky Department of Education 
and the Virginia Department of Education. In 
addition, the Common Core of Data (data sets 
maintained by the U.S. Department of Educa- 
tion’s National Center for Education Statistics) 
was used for information on school character- 
istics. The data covered 2001/02-2005/06 in 
Kentucky and 2001/02-2004/05 in Virginia, 
so no pre- and post-NCLB comparisons can be 
made. 

The findings of this report are: 

1. Kentucky schools experienced upward 
trends in the percentage of students pass- 
ing between 2001/02 and 2005/06, with 
increases in both the percentage testing 
proficient and the percentage testing 
advanced. Virginia schools also experi- 
enced upward trends in the percentage 
passing between 2001/02 and 2004/05, but 
in grade 5 reading and math and grade 3 
math a greater share of students moved 
from proficient to advanced than from 
below proficient to proficient. 



2. Positive associations were found between 
school-level changes in the percentage 
passing and changes in the percentage 
testing advanced in Kentucky and Vir- 
ginia schools in both reading and math in 
all grades tested. 

3. In most subjects and grades the posi- 
tive association between school-level 
changes in the percentage passing and 
in the percentage testing advanced holds 
across schools regardless of whether the 
percentage of students passing in 2001/02 
was above or below the 2004/05 annual 
measurable objective level. Even in schools 
where the 2001/02 percentage of students 
passing was below the 2004/05 annual 
measurable objective, schools whose 
percentage passing rate increased also saw 
an increase in the percentage testing ad- 
vanced (with the exception of high school 
end of course reading exams in Virginia). 

4. Controlling for school characteristics such 
as 2001/02 level and change in the percent- 
age of students eligible for free or reduced- 
price lunch, 2001/02 level and change 

in percentage of racial/ethnic minority 
students, and locale accounts for some of 
the variation in changes in the percentage 
testing advanced but does not alter the as- 
sociation between changes in the percent- 
age passing and changes in the percentage 
testing advanced in Kentucky schools 
and does not eliminate the association in 
Virginia schools. 

It is not possible to infer from this report’s 
findings any conclusions regarding the factors 
that brought about the observed relationships. 
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The No Child Left Behind 
Act of 2001 requires 
states to test students in 
reading and math and 
identify them as below 
proficient, proficient, or 
advanced. Schools are 
held accountable only for 
ensuring that students 
test proficient or better 
(both considered passing), 
leading to concerns that 
a focus on increasing the 
percentage of students 
testing proficient on state 
assessments may have an 
unintended consequence 
of reducing — or not 
increasing — the percentage 
of students testing 
advanced. Analysis of 
the data in Kentucky and 
Virginia finds that schools 
with the greatest increases 
in the percentage passing 
also have the greatest 
increases in the percentage 
testing advanced. 



WHY THIS STUDY? 

The No Child Left Behind (NCLB) Act of 2001 re- 
quires states to test all students in reading and math 
and identify them as below proficient, proficient, or 
advanced, but the law holds schools accountable only 
for ensuring that students test proficient or better. 
Some researchers and commentators have suggested 
that because the NCLB Act focuses on schools that 
fail to meet standards for moving students from 
below proficient to proficient, schools may give more 
attention to students testing below proficient at the 
expense of those testing proficient or above (Booher- 
Jennings 2005; de Vise 2007). 1 A contrasting view 
holds that steps taken to improve the performance 
of students testing below proficient are likely to 
improve results for students testing proficient or 
advanced (see box 1 for definitions of key terms). 

Concern about the impact on students testing 
proficient and above from the NCLB Act’s focus 
on improving the performance of students testing 
below proficient has numerous implications. State 
officials, district and school leaders, and others 
have asked whether attention to proficiency alone 
might have unintended consequences for students 
testing proficient or advanced. Policymakers are 
interested in understanding how test scores move 
together. Some superintendents and principals in 
Kentucky and Virginia have identified a need for 
support in developing and interpreting statistics 
that provide a balanced, accurate, and multidi- 
mensional view of how well schools are serving 
students testing below proficient, proficient, and 
advanced (Sheekey, Bausch, and Peterson 2008a, 
2008b). To better inform decisions about what 
kinds of performance measures and standards are 
needed, policymakers at the state level are inter- 
ested in understanding how the distributions of 
students at each proficiency level vary over time. 

A growing body of literature is examining assess- 
ments and the impact of high-stakes accountability 
systems (such as NCLB) on student achievement — 
especially trends in test results disaggregated by 
achievement level — as well as the association among 
them. Formal tests of the effect of accountability 
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PERCENTAGE OF STUDENTS PASSING & PERCENTAGE TESTING ADVANCED ON KENTUCKY & VIRGINIA ASSESSMENTS 



BOX 1 

Definitions of key terms 

Not meeting the 2004/05 annual 
measurable objective in 2001/02. The 
No Child Left Behind (NCLB) Act 
was enacted in 2001/02, whereas 
2004/05 was the first year in which 
the annual measurable objectives 
increased in Kentucky and Virginia. 
Schools whose percentage of students 
passing in 2001/02 was below the 
2004/05 annual measurable objective 
would need to increase that percent- 
age to meet the objective in 2004/05. 
In Kentucky the annual measur- 
able objective varies by subject and 
school level: for elementary schools 
it is 53.86 in reading and 32.14 in 
math; for middle schools it is 52.40 
in reading and 26.93 in math; for 
high schools it is 29.35 in reading and 
29.79 in math. In Virginia the annual 
measurable objective for both reading 
and math in 2004/05 was 70. 



Meeting the 2004/05 annual measurable 
objective in 2001/02. Schools with a per- 
centage passing rate in 2001/02 equal 
to or greater than the 2004/05 annual 
measurable objective would need only 
to maintain that rate to meet the an- 
nual measurable objective in 2004/05. 

Advanced. Under NCLB-mandated 
accountability, the highest level of stu- 
dent performance on the state assess- 
ment used for NCLB accountability. 

In Kentucky the “distinguished” level 
is synonymous with advanced. 

Annual measurable objective. The 
state-set level for the percentage of 
students expected to test proficient or 
above under NCLB accountability in 
a given year. In all states the annual 
measurable objective must reach 100 
percent by 2013/14. 

Annualized change. The average year 
to year change in the percentage 



of students passing or the percent- 
age testing advanced over the study 
period (2001/02-2005/06 in Kentucky 
and 2001/02-2004/05 in Virginia). 

Below proficient. The level of student 
performance on the state test used 
for NCLB accountability below the 
proficient level. Kentucky uses the 
terms “novice” and “apprentice” lev- 
els instead of “below proficient”; both 
have been combined into the below 
proficient category for this study. 

Passing. The level of student per- 
formance on the state test used for 
NCLB accountability at or above the 
state-defined proficient cutscore. 

Proficient. Under NCLB-mandated 
accountability, the level of student 
performance required for achieving 
proficiency on the state assessment 
used for NCLB accountability, but not 
performance at the advanced level. 



systems on the rate of improvement of test scores 
across levels of proficiency have provided mixed 
evidence (for example, Neal and Schanzenbach 2007; 
Springer 2008; Center on Education Policy 2008). 
Several studies have found that gains in achieve- 
ment are highest for students at or below the passing 
cutscore, while more advanced students experience 
small or negative changes in achievement (Deere 
and Strayer 2001; Krieg 2008; Neal and Schanzen- 
bach 2007; Reback 2008). Other studies have found 
that test score gains by low-performing students do 
not come at the expense of more advanced students 
(Ballou, Liu, and Rolle 2006; Ballou and Springer 
2008; Springer 2008). Because previous studies fo- 
cused on student-level changes in achievement, this 
study contributes to the literature by examining the 
changes in the percentage of students passing and 
percentage testing advanced at the school level. The 
report makes no claims about what is happening at 
the school or classroom level to drive those changes. 



The intent of this study is to examine whether a 
statistical association exists between changes in 
the percentage of students passing and simultane- 
ous changes in the percentage of students testing 
advanced. Four questions drive the analysis: 

1. What are the overall school-level trends in 
the percentage of students passing (testing 
proficient or advanced) and the percentage 
testing advanced on state assessment tests in 
Kentucky and Virginia? 

2. What is the statistical association between 
annualized changes in the school-level per- 
centage of students passing and annualized 
changes in the school-level percentage testing 
advanced? 

3. Does this association vary when controlling 
for the percent of students passing in 2001/02? 



WHY THIS STUDY? 
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BOX 2 

Methodology 

The report examines overall school- 
level trends in the percentage of 
students passing (testing proficient or 
advanced) and the percentage testing 
advanced on state assessments used for 
No Child Left Behind (NCLB) account- 
ability in Kentucky and Virginia, the 
statistical association between annual- 
ized changes in the percentage passing 
and in the percentage testing ad- 
vanced, and the association controlling 
for the percentage passing in 2001/02. 

Data sources. Three data sources were 
used: 

• The Kentucky Department of 
Education (2007) provided 
school-level data on the number 
of students tested overall, the per- 
centages of students testing below 
proficient (novice or apprentice), 
proficient, and above proficient 
(distinguished) for reading in 
grades 4, 7, and 10 and math in 
grades 5, 8, and 11 for 2001/02- 
2005/06. The percentage passing 
rate is calculated by adding the 
percentage testing proficient and 
the percentage testing advanced. 

• The Virginia Department of 
Education (2007) provided 
school-level data, by grade tested, 
on the number of students 



tested overall, the percentages of 
students testing below proficient, 
proficient, and advanced in 
reading and math for 2001/02— 
2004/05. Grades 3, 5, and 8 were 
tested in elementary and middle 
schools, and end of course exams 
were given in high school read- 
ing and Algebra 2. 

• The Common Core of Data (U.S. 
Department of Education 2007), 
which contains information on 
public school student demo- 
graphic and socioeconomic char- 
acteristics by school, provided 
data on total enrollment, the 
percentage of students eligible 
for free or reduced-price lunch, 
the percentage of racial/ethnic 
minority students, and school 
locale for 2001/02 and 2005/06 
for both Kentucky and Virginia. 

Analysis. Analysis occurred in five 
stages. The overall statewide trends 
in the percentage of students passing 
and the percentage testing advanced 
were examined by state, subject, and 
grade. Schools were grouped into four 
categories by deviations from the 
mean change in percentage passing 
and compared for average change 
in percentage testing advanced. 
Changes in the percentage testing 
advanced were then regressed on 
changes in the percentage passing. 
The sample was split into two groups 



based on the percentage of students 
passing in 2001/02 — schools whose 
2001/02 percentage passing was 
above the 2004/05 annual measurable 
objective and schools whose 2001/02 
percentage passing was below the 
2004/05 objective— and the regres- 
sion was run again to see whether 
schools threatened with NCLB sanc- 
tions showed a different association 
than other schools (see appendix B). 
Finally, a multivariate ordinary least 
squares regression was run to control 
for the percentage testing proficient 
in 2001/02, the 2001/02 level and sub- 
sequent change in percentage eligible 
for free or reduced-price lunch, the 
2001/02 level and subsequent change 
in the percentage of racial/ethnic 
minority students, and school locale. 

The sample was limited to schools 
that tested enough students in each 
grade in every year of the study 
period for NCLB accountability ac- 
cording to each state’s accountability 
workbook. The minimum number is 
10 in Kentucky and 50 in Virginia. 
Thus the sample includes schools in 
Kentucky that tested at least 10 stu- 
dents per grade tested every year dur- 
ing 2001/02-2004/05 (see appendix A 
for details about the sample schools 
and the excluded schools). 

All analyses were conducted using 
school-level data by state, subject, 
and grade. 



4. Does this association vary when controlling 
for characteristics of a school’s students, such 
as level and change in the percentage of stu- 
dents eligible for free or reduced-price lunch, 
level and change in the percentage of racial/ 
ethnic minority students, and the school’s 
locale (urban, suburban, town, or rural)? 



This study is descriptive and makes no claims 
about causality. It uses only post-NCLB data and 
therefore can make no claims about the effect of 
NCLB on changes in the distribution of proficiency 
at schools. Furthermore, the observed changes may 
have occurred in the absence of NCLB or may be 
associated with demographic or cultural change. 



4 



PERCENTAGE OF STUDENTS PASSING & PERCENTAGE TESTING ADVANCED ON KENTUCKY & VIRGINIA ASSESSMENTS 



The study examines the association between 
changes in the percentage of students passing and 
the percentage testing advanced over a limited 
period of time. The study methodology is summa- 
rized in box 2 and described fully in appendix A. 



KENTUCKY AND VIRGINIA SCHOOLS SHOW 
UPWARD TRENDS IN PERCENTAGE OF STUDENTS 
PASSING AND PERCENTAGE TESTING ADVANCED 

This section describes the trend in average percent- 
age of students passing disaggregated by the per- 
centage testing proficient and the percentage testing 
advanced. The number of schools in the sample by 
state, grade, and subject are presented in table 1. 

The results in this section are based on average 
school-level test results over a statewide sample. As 
a result, they may mask significant variation across 
schools. For example, it is possible that individual 
schools that show the largest increases in the per- 
centage of students passing show no increase or a 
decrease in the percentage testing advanced. 



Kentucky 

A greater share of students tested at the proficient 
level than at the advanced level in both 2001/02 
and 2005/06 (figure 1). In each subject and grade 
tested, a larger percentage of students tested at 
the proficient level in 2005/06 than in 2001/02. 
The same trend was observed for students at the 
advanced level. 



In Kentucky a greater 
share of students tested 
at the proficient level 
than at the advanced 
level in both 2001/02 
and 2005/06, and in 
Virginia the percentage 
of students passing both 
subjects increased in all 
grade levels between 
2001/02 and 2004/05 



The increase in the percentage 
of students passing ranges from 
7 points in grade 7 reading to 
21 points in grade 5 math. The 
increases in the percentage testing 
advanced are greater than the 
increases in the percentage testing 
proficient in reading for grades 

4 and 10 and in math for grades 

5 and 11 (in terms of both point 
increases and the share of the in- 
crease in the percentage of students 



TABLE 1 

Number of schools in sample for Kentucky 
(2001/02-2005/06) and Virginia, by grade and 
subject (2001/02-2004/05) 




Source: Authors' calculations based on data from Kentucky Department 
of Education (2007) and Virginia Department of Education (2007). 



passing). In grade 7 reading and grade 8 math more 
of the increase in the percentage of students passing 
comes from increases in the percentage testing 
proficient than in those testing advanced. Overall, 
schools in Kentucky experienced upward shifts in 
the percentage of students passing coming from 
increases in both the percentage testing proficient 
and the percentage testing advanced. 



Virginia 



Virginia schools show an increase in the percent- 
age of students passing in all grade levels and both 
subjects between 2001/02 and 2004/05 (figure 2). 2 
The increases range from a low of 4 points in grade 
3 reading and high school reading end of course 
tests to a high of 13 points on the Algebra 2 end of 
course test. 

For grade 3 reading, high school reading, and 
grade 8 math, changes in the percentage testing 
proficient make up a greater share of the increase 
in the percentage of students passing than do 
changes in the percentage testing advanced. For 
grade 8 reading and the high school Algebra 2 end 
of course test, both the percentage testing profi- 
cient and percentage testing advanced increase, 
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FIGURE 1 

Percentage of students passing in Kentucky schools, by percentage testing proficient or advanced, 2001/02 
and 2005/06 
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Source: Authors' calculations based on data from Kentucky Department of Education (2007). 



FIGURE 2 

Percentage of students passing in Virginia schools, by percentage testing proficient or advanced, 2001/02 
and 2004/05 
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(end of course exam) Algebra 2 

Reading Math (end of course exam) 

Source: Authors' calculations based on data from Virginia Department of Education (2007). 



but a greater share of the increase in the percent- 
age of students passing comes from the change in 
the percentage testing advanced. 

Grade 5 reading and math and grade 3 math, 
however, show a different pattern. The increase 
in the percentage of students passing comes from 



an increase in the percentage testing advanced; 
there is no change (or a decrease) in the percentage 
testing proficient. For grade 5 reading and grade 
3 math the percentage testing proficient declines 
by 5 points, but the percentage testing advanced 
increases by 12 points, for a 7-point net increase in 
percentage of students passing. For grade 5 math 
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all of the increase in the percentage of students 
passing comes from the 10-point increase in the 
percentage testing advanced with no change in 
the percentage testing proficient. Thus, for grade 
5 reading and math and grade 3 math, the rise 
in the percentage of students passing reflects a 
greater share of students moving from proficient to 
advanced than from below proficient to proficient. 



SCHOOLS WITH THE LARGEST INCREASES IN 
THE PERCENTAGE OF STUDENTS PASSING 
SHOW THE LARGEST INCREASES IN THE 
PERCENTAGE TESTING ADVANCED 

This section takes a closer look at the relationship 
between annualized changes in the school-level 
percentage of students passing and annualized 
changes in the school-level percentage testing ad- 
vanced by considering variation in the percentage 
of students passing. The analysis can show whether 
the same trend holds across all schools or whether 
the association differs across schools with different 
changes in the percentage of students passing. 

For this analysis, schools were split into four groups 
based on their average change in the percentage of 
students passing: schools with the smallest (and 
sometimes negative) changes, schools with below av- 
erage increases, schools with above average increases, 
and schools with the largest increases. The average 
change in the percentage testing advanced was then 
compared across the four groups. (See appendix B for 
details of how the groups were constructed.) 

The results produced a consistent finding: schools 
with the largest increases in the percentage of 
students passing also have the largest increases in 

the percentage testing advanced. Ex- 
amination of each subject and grade 
for both states showed a consistent 
pattern of larger average annualized 
changes in the school-level percent- 
age of students testing advanced as 
the change in the percentage passing 
grew larger (see figures B1-B6 and 
B9-B16 in appendix B). 



Schools with the 
largest increases in the 
percentage of students 
passing also have 
the largest increases 
in the percentage 
testing advanced 



FIGURE 3 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 5 math in 
Virginia schools grouped by standard deviation, 
2001/02-2004/05 



Annualized change in percentage testing advanced 



5.60 




0.21 



More than 1 Within 1 Within 1 More than 1 

standard deviation standard deviation standard deviation standard deviation 
below the mean below the mean above the mean above the mean 
(n = 92) (n = 308) (n = 234) (n = 105) 



Annualized change in percentage passing 

Source: Authors' calculations based on data from Virginia Department 
of Education (2007). 



Figure 3 illustrates this trend. For each group for 
grade 5 math in Virginia, schools with the largest in- 
crease in the percentage of students passing show the 
largest annualized increase in the percentage testing 
advanced, and schools with the smallest increase 
(or a decrease) in the percentage passing show the 
smallest annualized change in the percentage testing 
advanced. The same pattern is found in reading and 
math at all grade levels in Kentucky and Virginia, 
with the exception of grade 5 reading in Virginia. 

In all cases schools with the smallest changes in the 
percentage of students passing (more than one stan- 
dard deviation below the mean) show the smallest 
changes in the percentage testing advanced. 



SCHOOL-LEVEL CHANGES IN THE PERCENTAGE 
OF STUDENTS TESTING ADVANCED ARE 
POSITIVELY ASSOCIATED WITH CHANGES 
IN THE PERCENTAGE PASSING 



This section refines the previous estimates by 
calculating the statistical associations between 
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school-level changes in the percentage of students 
passing and changes in the percentage testing 
advanced. Bivariate regression analysis was used to 
further examine the association between annualized 
changes in the school-level percentage of students 
passing and annualized changes in the school-level 
percentage testing advanced (see appendix B for 
detailed scatterplots and results). In the regression 
analysis the school-level change in percentage testing 
advanced is modeled as a function of change in the 
percentage passing. The slope coefficient represents 
the direction of the relationship — how a 1 percent- 
age point increase in the percentage of students 
passing is associated with a change in the percentage 
testing advanced. Positive slope coefficients mean 
that increases in the percentage of students pass- 
ing are associated with increases in the percentage 
testing advanced. The strength of the relationship is 
represented by the r 2 value, with higher values indi- 
cating that the association accounts for a larger share 
of the variation in the percentage testing advanced. 

A statistically significant positive association is 
found for all subjects and grades in both Kentucky 
and Virginia (table 2). In Kentucky the magnitude 
of change in the percentage of students testing 
advanced based on the change in the percentage 
passing is similar across grades, ranging from 
.28 to .52. That means that on average for every 
1 percentage point increase in the percentage of 
students passing, the percentage of students test- 
ing advanced increases a third to a half a percent- 
age point. In Virginia the variation in magnitude 
across schools is greater, ranging from 0.16 to 0.89. 

The association is strongest in Kentucky for grade 
10 reading, grade 8 math, and grade 11 math 
and in Virginia for grade 3 math. The associa- 
tion is weakest in Kentucky for grade 4 reading 
and in Virginia for grade 5 reading. The associa- 
tion is also weak in Virginia for grade 8 reading 
and math and the Algebra 2 end of course exam. 

As the slope coefficients and r 2 values show, the 
overall association between school-level changes 
in the percentage of students passing and changes 
in the percentage testing advanced is stronger in 
Kentucky than in Virginia (see table 2). 



TABLE 2 

Ordinary least squares regressions for the 
association between school-level changes in the 
percentage of students testing advanced and 
changes in the percentage passing in Kentucky 
(2001/02-2005/06) and Virginia (2001/02-2004/05) 



State, subject, 
and grade 


Number 
of schools 


Slope 

coefficient 


r 2 


Kentucky 


Reading 


Grade 4 


616 


0.282*** 


.170 


Grade 7 


202 


0.324*** 


.399 


Grade 10 


197 


0.524*** 


.502 


Math 


Grade 5 


604 


0.352*** 


.312 


Grade 8 


202 


0.350*** 


.510 


Grade 11 


197 


0.400*** 


.434 


Virginia 


Reading 


Grade 3 


710 


0.404*** 


.280 


Grade 5 


737 


0.162*** 


.035 


Grade 8 


323 


0.323*** 


.109 


High school reading 








(end of course exam) 


270 


0.793*** 


.273 


Math 


Grade 3 


723 


0.887*** 


.420 


Grade 5 


739 


0.364*** 


.223 


Grade 8 


321 


0.260*** 


.167 


High school Algebra 2 








(end of course exam) 


232 


0.374*** 


.168 



*** Significant at p < .001 . 

Source: Authors' calculations based on data from Kentucky Department 
of Education (2007) and Virginia Department of Education (2007). 



ASSOCIATIONS REMAIN POSITIVE BUT ARE 
WEAKER IN SCHOOLS WHERE THE PERCENTAGE 
PASSING IN 2001/02 IS BELOW THE 2004/05 
ANNUAL MEASURABLE OBJECTIVE 

Under the NCLB Act each state defines its own an- 
nual measurable objective for test results by level, 
with the goal of achieving 100 percent proficiency 
by 2013/14. Each state established a starting an- 
nual measurable objective for each subject and 
grade in 2001/02. In both Kentucky and Virginia 
the annual measurable objectives remained the 
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Even among schools 
whose percentage of 
students passing in 
2001/02 was below 
the 2004/05 annual 
measurable objective, the 
statistically significant 
positive association 
between changes in 
percentage passing and 
changes in percentage 
testing advanced holds 



same for three years, then in- 
creased in 2004/05 and remained 
the same for three years. 3 



Schools whose percentage of 
students passing in 2001/02 is 
below the 2004/05 annual mea- 
surable objective might be more 
likely than schools whose percent- 
age passing is above the 2004/05 
objective to focus on moving 
students from below proficient to 
proficient than from proficient to 
advanced. Therefore, the associa- 
tion between changes in the percentage of students 
passing and the percentage testing advanced was 
compared for schools in the two groups. 



In Kentucky relatively few schools (7-27 percent, 
depending on subject and grade) had a percentage 
of students passing in 2001/02 below the 2004/05 
annual measurable objective. And in Virginia fewer 
than 5 percent of high schools had a percentage pass- 
ing on the end of course reading test in 2001/02 below 
the 2004/05 annual measurable objective. But close to 
a quarter of Virginia schools were below the 2004/05 
annual measurable objective for grade 5 reading 
and high school Algebra 2 (end of course test), and 
approximately 40-50 percent of schools were below 
the 2004/05 annual measurable objective for grades 
3 and 8 reading and grades 5 and 8 math. Compar- 
ing the “below” and “above” groups tests whether 
the results hold after controlling for the school-level 
percentage of students passing in 2001/02. As in the 
preceding section, the analysis is based on regres- 
sions of the annualized change in the school-level 
percentage of students testing advanced on the annu- 
alized change in the school-level percentage passing 
(see appendix A for a discussion of the methods and 
variables, and appendix B for detailed results). 



The analysis shows that even among schools whose 
percentage of students passing in 2001/02 was 
below the 2004/05 annual measurable objective, 
the statistically significant positive association be- 
tween changes in percentage passing and changes 
in percentage testing advanced holds (table 3), 



with the exception of the high school end of course 
reading test in Virginia. In each case, however, the 
slope coefficients are lower for the “below” schools 
than for the “above” schools, meaning that in 
schools where the percentage of students passing in 
2001/02 was below the 2004/05 annual measurable 
objective, the increases in the percentage testing 
advanced was smaller on average for a one percent- 
age point increase in the percentage passing. 

In Virginia for schools whose percentage of stu- 
dents passing in 2001/02 was below the 2004/05 
annual measurable objective, the explained vari- 
ance in the change in the percentage of students 
testing advanced is greater (as measured by the 
r 2 value) for elementary schools than for middle 
and high schools. As with the analysis of the full 
sample, the explained variation is greater in Ken- 
tucky schools than in Virginia schools. 



POSITIVE ASSOCIATIONS HOLD 
AFTER CONTROLLING FOR SCHOOL- 
LEVEL CHARACTERISTICS 

Other school factors, such as demographic compo- 
sition, might account for the association between 
changes in the percentage of students passing and 
changes in the percentage testing advanced. A 
regression analysis of changes in the percentage of 
students testing advanced was conducted to control 
for multiple factors, including the percentage testing 
proficient 4 in 2001/02, the 2001/02 level and subse- 
quent change in the percentage of students eligible 
for free or reduced-price lunch and in the percent- 
age of racial/ethnic minority students, and the 
school’s locale (urban, suburban, town, or rural). 

The analysis shows that positive, statistically 
significant associations between changes in the 
percentage of students passing and in the percent- 
age testing advanced remain after adding the con- 
trols. It also finds a positive, statistically significant 
association between the percentage of students 
testing proficient in 2001/02 and changes in the 
percentage testing advanced for grades 4 and 10 
reading and grades 5 and 11 math in Kentucky and 
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TABLE 3 

Ordinary least squares regressions of school-level changes in the percentage of students testing advanced 
on changes in the percentage passing for schools with the percentage passing in 2001/02 below and above 
the 2004/05 annual measurable objective 





Percentage passing in 2001/02 
below the 2004/05 annual 
measurable objective 


Percentage passing in 2001/02 
above the 2004/05 annual 
measurable objective 


State, subject, and grade 


Number of 
schools 


Slope 

coefficient 


r 2 


Number of 
schools 


Slope 

coefficient 


r 2 


Kentucky 


Reading 


Grade 4 


45 


0.208*** 


.344 


571 


0.327*** 


.192 


Grade 7 


25 


0.235** 


.300 


177 


0.343*** 


.417 


Grade 10 


55 


0.429*** 


.577 


142 


0.633*** 


.610 


Math 


Grade 5 


65 


0.346*** 


.426 


539 


0.378*** 


.328 


Grade 8 


55 


0.235*** 


.344 


147 


0.388*** 


.584 


Grade 11 


49 


0.379*** 


.494 


148 


0.573*** 


.536 


Virginia 


Reading 


Grade 3 


271 


0.312*** 


.233 


439 


0.537*** 


.199 


Grade 5 


160 


0.290*** 


.151 


577 


0.396*** 


.097 


Grade 8 


167 


0.229*** 


.100 


156 


0.725*** 


.234 


High school reading (end of course exam) 


12 


0.445 


.086 


258 


0.826*** 


.240 


Math 


Grade 3 


114 


0.806*** 


.452 


609 


1.062*** 


.362 


Grade 5 


298 


0.366*** 


.275 


441 


0.658*** 


.261 


Grade 8 


145 


0.235*** 


.231 


176 


0.649*** 


.262 


High school Algebra 2 (end of course exam) 


59 


0.295* 


.108 


173 


0.838*** 


.320 



* Significant at p < .05;** significant at p < .01;*** significant at p < .001. 

Source: Authors' calculations based on data from Kentucky Department of Education (2007) and Virginia Department of Education (2007). 



grades 3, 5, and 8 reading and grade 5 math in Vir- 
ginia. Schools with a higher percentage of students 
testing proficient in 2001/02 are associated with 
larger increases in the percentage testing advanced. 
No consistent associations are found between the 
school demographic controls and the change in the 
percentage of students testing advanced. (See tables 
in appendix C for detailed results.) 



LIMITATIONS OF THE STUDY 

Drawing inferences from the analysis of trends in 
assessment data is complex and challenging. This 



section discusses several limitations that affected 
this study and informed the interpretation of the 
results. 

• The analysis describes only the association 
at the school level between changes in the 
percentage of students passing and changes 
in the percentage testing advanced. It cannot 
identify what practices, if any, drive these 
changes at the schools. Furthermore, because 
the analysis is conducted at the school level, it 
does not follow cohorts of students over time 
and cannot answer how much improvement 
individual students show. 
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The analyses presented 
here indicate that 
increases in the 
percentage of students 
testing at proficient 
levels in math and 
reading is not associated 
with a decline in 
those achieving at 
advanced levels 



• The study analyzed data over a 
fixed period in two states. The 
results cannot be extrapolated 
beyond the period studied or 
the states examined. As the 
state accountability systems 
change, the associations may 
change. Similarly, lack of data 
for the pre-NCLB period pre- 
vented comparison of trends 
before and after the NCLB Act 
was passed, which would have 
indicated whether there were changes associated 
with the inception of the NCLB Act. 



• The analysis examined all students tested 
by state, subject, and grade. It did not disag- 
gregate students by NCLB subgroup (racial/ 
ethnic groups, students with disabilities, 
economically disadvantaged students, and 
English language learner students), and the 
patterns may differ for the subgroups. 



• Demographic change within schools over the 
period studied could generate bias. The multi- 
variate models controlled for the observable 
characteristics of the percentage of students 
eligible for free or reduced-price lunch and 
percentage of racial/ethnic minority students 
in the baseline year and changes in those char- 
acteristics over the study period and found no 
differences in the associations examined. 



• The sample excluded small schools that did 
not test enough students to come under NCLB 
accountability, so the results may not be gen- 
eralizable to small schools. 



CONCLUSIONS 

The picture that emerges from this study is that, 
in general, schools with the smallest increases in 



the percentage of students passing in reading and 
math have the smallest increases in the per- 
centage testing advanced, and schools with the 
largest increases in the percentage passing have 
the largest increases in the percentage testing 
advanced. 

There is a positive statistical association between 
changes in the percentage of students passing and 
changes in the percentage testing advanced, but 
the strength of the association varies somewhat 
depending on the subject, the grade, and whether 
the percentage of students passing in 2001/02 was 
below or above the 2004/05 annual measurable 
objective. The association is stronger in Kentucky 
than in Virginia, and stronger in elementary 
schools than in middle or high schools. 

School characteristics, such as the percentage of 
students testing proficient in 2001/02, the percent- 
age of students eligible for free or reduced-price 
lunch, the percentage of racial/ethnic minority 
students, and school locale, improved the explana- 
tion of variation in changes in the percentage of 
students testing advanced. School characteristics 
did not, however, alter the fundamental positive 
relationship between changes in the percentage of 
students passing and changes in the percentage 
testing advanced. 

While correlation and regression analyses show a 
consistently positive association between trends 
in students testing at different levels, some 
schools exhibit different behavior. What moves 
test scores — and whether different categories 
move together — is an extremely complex ques- 
tion. The analyses presented here indicate that 
increases in the percentage of students testing 
at proficient levels in math and reading is not 
associated with a decline in those achieving at 
advanced levels. In fact, it is more likely that in- 
creases in the passing and advanced levels move 
together. 
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NOTES 

1 . 



2 . 



This report uses the National Assessment of 
Educational Progress classification terminol- 
ogy of “below proficient,” “proficient,” and 
“advanced.” Virginia uses these terms, but 
Kentucky uses the term “distinguished” 
instead of “advanced.” Kentucky also has two 
below proficient levels: “novice,” the lowest 
level, and “apprentice.” This report combines 
these two levels and refers to them as “below 
proficient.” 

While Virginia administers all tests at the end 
of the year, it refers to the high school tests 
as end of course tests. Students typically take 
the high school reading test in grade 10. End 
of course exams are given for both Geometry 
and Algebra 2 in high school, but only Algebra 
2 is examined here for ease of presentation. 
(Results for Geometry are available from the 
authors on request.) 

This pattern refers to the original Con- 
solidated State Accountability Workbooks. 



Virginia revised its annual measaurable 
objectives in June 2005 so that instead of 
increasing then remaining the same for three 
years, the objectives would increase incremen- 
tally each year after 2004/05. 

4. The analysis controlled for the percentage of 
students testing proficient in 2001/02 because 
schools with a high percentage of proficient 
students might be more likely to increase the 
percentage of students testing at the advanced 
level than either schools with many below 
proficient students or schools with many 
students already testing advanced. NCLB 
accountability systems compare successive 
cohorts of students entering each grade, not 
the same cohort of students over time. But 
because the test scores of students entering 
a given grade change slowly over time, in 
general, schools with many proficient stu- 
dents at one point find it “easier” to increase 
the percentage testing advanced over time 
because their students need smaller increases 
in their test scores to move into the advanced 
category. 
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APPENDIX A 

DATA SOURCES AND METHODOLOGY 

This report examines school-level associations 
between the changes in the percentage of students 
passing (testing proficient or advanced) and changes 
in the percentage testing advanced on state assess- 
ments used for No Child Left Behind (NCLB) ac- 
countability in Kentucky and Virginia. It describes: 

• The average trends in percentage of students 
passing disaggregated by the percentage 
testing proficient and the percentage testing 
advanced over the early period of NCLB ac- 
countability in Kentucky (2001/02 to 2005/06) 
and Virginia (2001/02 to 2004/05). 

• The statistical association between annualized 
changes in the percentage of students passing 
and annualized changes in the percentage 
testing advanced at the school level. 

• The association for schools where the percent- 
age of students passing in 2001/02 was below 
the 2004/05 annual measurable objective and 
the association for schools where the percent- 
age of students passing in 2001/02 was above 
the 2004/05 objective (to account for schools 
likely to have difficulty achieving NCLB or 
state standards). 

• The association controlling for school charac- 
teristics such as the 2001/02 level and subse- 
quent changes in the percentage of students 
eligible for free or reduced-price lunch, the 
2001/02 level and subsequent changes in the 
percentage of racial/ethnic minority students, 
and the school’s locale (urban, suburban, 
town, or rural). 

Percentage passing is defined as the percentage 
of students per grade in a school testing profi- 
cient or advanced. The annualized change in the 
school-level percentage of students passing is the 
average yearly change over the period studied. 
Positive changes in the percentage passing reflect 
movement from the percentage of students testing 



below proficient to the percentage testing profi- 
cient or above. The percentage of students testing 
advanced is defined as the percentage of students 
per grade in a school testing advanced. The an- 
nualized change in percentage testing advanced is 
the average yearly change over the period studied 
(average annualized changes are used to smooth 
out any large year to year fluctuations). Positive 
changes in the percentage testing advanced reflect 
movement from the percentage of students testing 
proficient to the percentage testing advanced. Be- 
cause the analysis is conducted at the grade level 
within schools, it does not represent changes in 
individual students over time but rather it shows 
the overall performance of a school over time. 

All the analyses are based on schools with enough 
test-takers per grade tested to count for schoolwide 
NCLB accountability in each year of the study 
period. The minimum was 10 test-takers a year in 
Kentucky and 50 in Virginia. The following sec- 
tion compares schools in the sample and schools 
excluded from the sample. On average, excluded 
schools had lower total enrollments and lower 
percentages of students eligible for free or reduced- 
price lunch than did sample schools. Assessment 
results were not reported for students in schools 
where fewer than 10 students were tested. 



Data sources and sample 

The report uses three publicly available data 
sources: 

• The Kentucky Department of Education 

(2007) provided school-level data on the num- 
ber of students tested overall, the percentage 
testing below proficient (novice or apprentice), 
the percentage testing proficient, and the 
percentage testing above proficient (distin- 
guished) for reading and math for each year 
for 2001/02-2005/06. In the period studied, 
Kentucky tested reading in grades 4, 7, and 10 
and math in grades 5, 8, and 11. This report 
treats grades 4 and 5 as elementary school, 7 
and 8 as middle school, and 10 and 11 as high 
school. All percentages of students passing or 
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testing advanced for Kentucky in this study 
are based on these data. Data can be found 
online at ftp://ketsftp.kl2.ky.us/OAA. 

• The Virginia Department of Education (2007) 
provided school-level data, by grade tested, 
on the number of students tested overall, the 
percentage testing below proficient, the per- 
centage testing proficient, and the percentage 
testing above proficient (advanced) in reading 
and math for each year for 2001/02-2004/05. 
In the period studied, Virginia tested students 
in both reading and math in grades 3, 5, and 

8 and with end of course assessments in high 
school. End of course tests used in this report 
are reading and Algebra 2. The reading end 
of course test is typically taken at the end of 
grade 10. This report treats grades 3 and 5 as 
elementary school, 8 as middle school, and 
end of course reading and Algebra 2 as high 
school. All percentages of students passing or 
testing advanced for Virginia in this study are 
based on these data. Data can be found online 
at https://plpe.doe.virginia.gov/datareports/ 
assess_test_result.do. 

• The Common Core of Data (U.S. Depart- 
ment of Education 2007), which contains 
information on public school student demo- 
graphic and socioeconomic characteristics 
by school, provided data on total enrollment, 
the percentage of students eligible for free or 
reduced-price lunch, the percentage of racial/ 
ethnic minority students, and school locale 
for 2001/02 and 2005/06 for both Kentucky 
and Virginia. Data can be found online at 
http://nces.ed.gov/ccd/bat. 



Treatment of Kentucky data . Restricting the 
Kentucky sample to public schools with 10 or 
more test-takers in each year for 2001/02-2005/06 
reduced the overall sample by 14 percent for 
elementary schools, less than 1 percent for middle 
schools, and 2 percent for high schools. Table A1 
displays the total number of schools and those 
included in the sample. 

The restricted sample includes schools with higher 
total enrollment and lower percentages of students 
eligible for free or reduced-price lunch. Results 
may not generalize to small schools or schools 
with lower rates of students eligible for free or 
reduced-price lunch. Tables A2, A3, and A4 display 
comparisons of the mean and standard devia- 
tions of schools included in and excluded from the 
sample. Although the means differ, the standard 
deviations are large enough to have some overlap. 

Treatment of Virginia data. Virginia reports 
assessment data for schools with 10 or more 
test-takers, but only schools with 50 or more test- 
takers per school are held to NCLB accountability 
standards. Restricting the Virginia sample to 
public schools with 50 or more test-takers in each 
year for 2001/02-2004/05 resulted in retaining 78 
percent of reporting schools for grade 3 reading, 

76 percent for grade 3 math, 82 percent for grade 
5 reading, 79 percent for grade 5 math, 87 percent 
for grade 8 reading, 78 percent for grade 8 math, 

93 percent for the end of course assessments in 
reading, and 81 percent for the end of course as- 
sessments in Algebra 2. If the sample had included 
schools with 10-49 test-takers in each year for 
2001/02-2004/05, it would have retained 99 per- 
cent of schools that reported reading assessments 



TABLE A1 

Number of excluded and sample schools in Kentucky by grade and subject, 2001/02-2005/06 



School group 


Grade 4 
Reading 


Grade 5 
Math 


Grade 7 
Reading 


Grade 8 
Math 


Grade 10 
Reading 


Grade 11 
Math 


Excluded schools (fewer than 10 test-takers in any year) 


85 


97 


1 


1 


4 


4 


Sample schools (10 or more test-takers in each year) 


616 


604 


202 


202 


197 


197 


Total 


701 


701 


203 


203 


201 


201 



Source: Authors' calculations based on data from Kentucky Department of Education (2007). 
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TABLE A2 

Comparison of excluded and sample elementary schools in Kentucky, 2001/02-2005/06 





Excluded schools (fewer than 
10 test takers in any year) 


Sample schools (10 or more 
test takers in each year) 


Grade, subject, and characteristic 


Number 
of schools 


Mean 


Standard 

deviation 


Number 
of schools 


Mean 


Standard 

deviation 


Grade 4 reading 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


44 


67 


21 


596 


59 


23 


Percentage of racial/ethnic minority students, 2001/02 
(school) 


44 


9 


17 


596 


13 


17 


Total students at school, 2001/02 


44 


285 


207 


596 


416 


156 


Percentage of students testing below proficient, 2001/02 


39 


50 


26 


616 


39 


16 


Percentage of students testing proficient, 2001/02 


39 


46 


23 


616 


55 


13 


Percentage of students testing advanced, 2001/02 


39 


4 


6 


616 


6 


7 


Change in percentage testing below proficient, 2001/02- 
2005/06 


9 


-7 


27 


616 


-10 


14 


Change in percentage testing proficient, 2001/02-2005/06 


9 


-2 


36 


616 


4 


13 


Change in percentage testing advanced, 2001/02-2005/06 


9 


-3 


7 


616 


5 


10 


Grade 5 math 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


56 


67 


21 


584 


59 


23 


Percentage of racial/ethnic minority students, 2001/02 (school) 


56 


9 


17 


584 


13 


17 


Total students at school, 2001/02 


56 


285 


207 


584 


416 


156 


Percentage of students testing below proficient, 2001/02 


51 


73 


19 


584 


64 


17 


Percentage of students testing proficient, 2001/02 


51 


23 


17 


604 


29 


12 


Percentage of students testing advanced, 2001/02 


51 


3 


19 


604 


7 


7 


Change in percentage of students testing below proficient, 
2001/02-2005/06 


21 


-8 


19 


604 


-21 


15 


Change in percentage of students testing proficient, 
2001/02-2005/06 


21 


4 


14 


604 


10 


13 


Change in percentage of students testing advanced, 
2001/02-2005/06 


21 


3 


12 


604 


11 


9 



Source: Authors' calculations based on data from Kentucky Department of Education (2007) and National Center for Education Statistics (2007). 
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TABLE A3 

Comparison of excluded and sample middle schools in Kentucky, 2001/02-2005/06 





Excluded schools (fewer than 
10 test takers in any year) 


Sample schools (10 or more 
test takers in each year) 


Grade, subject, and characteristic 


Number 
of schools 


Mean 


Standard 

deviation 


Number 
of schools 


Mean 


Standard 

deviation 


Grade 7 reading 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


1 


75 


na 


197 


48 


19 


Percentage of racial/ethnic minority students, 2001/02 
(school) 


1 


26 


na 


197 


12 


15 


Total students at school, 2001/02 


1 


700 


na 


197 


587 


234 


Percentage of students testing below proficient, 2001/02 


1 


67 


na 


202 


44 


13 


Percentage of students testing proficient, 2001/02 


1 


30 


na 


202 


50 


10 


Percentage of students testing advanced, 2001/02 


1 


33 


na 


202 


6 


5 


Change in percentage testing below proficient, 2001/02- 
2005/06 


0 


na 


na 


202 


-7 


11 


Change in percentage testing proficient, 2001/02-2005/06 


0 


na 


na 


202 


5 


8 


Change in percentage testing advanced, 2001/02-2005/06 


0 


na 


na 


202 


1 


6 


Grade 8 math 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


1 


75 


na 


197 


48 


19 


Percentage of racial/ethnic minority students, 2001/02 
(school) 


1 


26 


na 


197 


12 


15 


Total students at school, 2001/02 


1 


700 


na 


197 


587 


234 


Percentage of students testing below proficient, 2001/02 


1 


92 


na 


202 


74 


12 


Percentage of students testing proficient, 2001/02 


1 


7 


na 


202 


21 


8 


Percentage of students testing advanced, 2001/02 


1 


8 


na 


202 


5 


12 


Change in percentage testing below, 2001/02-2005/06 


0 


na 


na 


202 


-9 


9 


Change in percentage testing proficient, 2001/02-2005/06 


0 


na 


na 


202 


5 


7 


Change in percentage testing advanced, 2001/02-2005/06 


0 


na 


na 


202 


4 


9 



na is not applicable. 

Source: Authors' calculations based on data from Kentucky Department of Education (2007) and National Center for Education Statistics (2007). 
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TABLE A4 

Comparison of excluded and sample high schools in Kentucky, 2001/02-2005/06 





Excluded schools (fewer than 
10 test takers in any year) 


Sample schools (10 or more 
test takers in each year) 


Grade, subject, and characteristic 


Number 
of schools 


Mean 


Standard 

deviation 


Number 
of schools 


Mean 


Standard 

deviation 


Grade 10 reading 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


4 


57 


8 


196 


38 


18 


Percentage of racial/ethnic minority students, 2001/02 
(school) 


4 


6 


12 


196 


10 


13 


Total students at school, 2001/02 


4 


532 


378 


196 


837 


404 


Percentage of students testing below proficient, 2001/02 


4 


82 


7 


197 


72 


11 


Percentage of students testing proficient, 2001/02 


4 


14 


5 


197 


21 


7 


Percentage of students testing advanced, 2001/02 


4 


5 


2 


197 


7 


5 


Change in percentage testing below, 2001/02-2005/06 


0 


na 


na 


197 


-13 


9 


Change in percentage testing proficient, 2001/02-2005/06 


0 


na 


na 


197 


4 


6 


Change in percentage testing advanced, 2001/02-2005/06 


0 


na 


na 


197 


9 


7 


Grade 11 math 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


4 


57 


8 


196 


38 


18 


Percentage of racial/ethnic minority students, 2001/02 
(school) 


4 


6 


12 


196 


10 


13 


Total students at school, 2001/02 


4 


532 


378 


196 


837 


404 


Percentage of students testing below proficient, 2001/02 


4 


82 


4 


197 


71 


12 


Percentage of students testing proficient, 2001/02 


4 


13 


2 


197 


21 


7 


Percentage of students testing advanced, 2001/02 


4 


6 


4 


197 


8 


6 


Change in percentage testing below proficient, 2001/02- 
2005/06 


0 


na 


na 


197 


-8 


8 


Change in percentage testing proficient, 2001/02-2005/06 


0 


na 


na 


197 


3 


6 


Change in percentage testing advanced, 2001/02-2005/06 


0 


na 


na 


197 


5 


5 



na is not applicable. 

Source: Authors' calculations based on data from Kentucky Department of Education (2007) and National Center for Education Statistics (2007). 
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TABLE A5 

Number of schools reporting on test-takers in Virginia by grade and subject, 2001/02-2004/05 



High school end 

Grade 3 Grade 5 Grade 8 of course exams 



School group 


Reading 


Math 


Reading 


Math 


Reading 


Math 


Reading 


Algebra 2 


Excluded schools 


9 or fewer test-takers in any year 


11 


48 


12 


50 


46 


79 


5 


13 


10-49 test-takers in each year 


191 


179 


152 


151 


3 


9 


16 


41 


Sample schools 


50 or more test-takers in each year 


710 


723 


737 


739 


323 


321 


279 


232 


Total 


912 


950 


901 


940 


372 


409 


300 


286 



Source: Authors' calculations based on data from Virginia Department of Education (2007). 



TABLE A6 

Comparison of excluded and sample elementary schools in Virginia, 2001/02-2004/05 





Excluded schools 
(10 49 test takers in any year) 


Sample schools (50 or more 
test takers in each year) 


Grade, subject, and characteristic 


Number 
of schools 


Mean 


Standard 

deviation 


Number 
of schools 


Mean 


Standard 

deviation 


Grade 3 reading 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


191 


47 


23 


710 


32 


22 


Percentage of racial/ethnic minority students, 2001/02 (school) 


191 


44 


33 


710 


38 


26 


Total students at school, 2001/02 


191 


375 


84 


710 


598 


162 


Percentage of students testing below proficient, 2001/02 


191 


32 


16 


710 


27 


14 


Percentage of students testing proficient, 2001/02 


191 


54 


11 


710 


56 


9 


Percentage of students testing advanced, 2001/02 


191 


14 


10 


710 


17 


11 


Change in percentage testing below proficient, 
2001/02-2004/05 


191 


-7 


16 


710 


-5 


12 


Change in percentage testing proficient, 2001/02-2004/05 


191 


5 


14 


710 


3 


11 


Change in percentage testing advanced, 2001/02-2004/05 


191 


3 


10 


710 


2 


9 


Grade 3 math 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


179 


47 


23 


723 


32 


22 


Percentage of racial/ethnic minority students, 2001/02 (school) 


179 


42 


32 


723 


38 


27 


Total students at school, 2001/02 


179 


372 


86 


723 


596 


163 


Percentage of students testing below proficient, 2001/02 


179 


23 


15 


723 


19 


11 


Percentage of students testing proficient, 2001/02 


179 


41 


10 


723 


40 


9 


Percentage of students testing advanced, 2001/02 


179 


35 


16 


723 


41 


16 


Change in percentage testing below proficient, 
2001/02-2004/05 


179 


-10 


14 


723 


-7 


10 


Change in percentage testing proficient, 2001/02-2004/05 


179 


-3 


14 


723 


-5 


10 


Change in percentage testing advanced, 2001/02-2004/05 


179 


13 


17 


723 


12 


14 



(CONTINUED) 





18 



PERCENTAGE OF STUDENTS PASSING & PERCENTAGE TESTING ADVANCED ON KENTUCKY & VIRGINIA ASSESSMENTS 



TABLE A6 (CONTINUED) 

Comparison of excluded and sample elementary schools in Virginia, 2001/02-2004/05 





Excluded schools 
(10 49 test takers in any year) 


Sample schools (50 or more 
test takers in each year) 


Grade, subject, and characteristic 


Number 
of schools 


Mean 


Standard 

deviation 


Number 
of schools 


Mean 


Standard 

deviation 


Grade 5 reading 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


152 


45 


23 


737 


32 


22 


Percentage of racial/ethnic minority students, 2001/02 (school) 


152 


43 


32 


737 


37 


26 


Total students at school, 2001/02 


152 


364 


79 


737 


588 


162 


Percentage of students testing below proficient, 2001/02 


152 


24 


15 


737 


22 


13 


Percentage of students testing proficient, 2001/02 


152 


60 


11 


737 


61 


8 


Percentage of students testing advanced, 2001/02 


152 


16 


10 


737 


18 


10 


Change in percentage testing below proficient, 
2001/02-2004/05 


152 


-9 


14 


737 


-8 


9 


Change in percentage testing proficient, 2001/02-2004/05 


152 


-2 


13 


737 


-4 


12 


Change in percentage testing advanced, 2001/02-2004/05 


152 


11 


10 


737 


12 


9 


Grade 5 math 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


151 


45 


23 


739 


32 


22 


Percentage of racial/ethnic minority students, 2001/02 (school) 


151 


44 


32 


739 


37 


27 


Total students at school, 2001/02 


151 


362 


80 


739 


587 


162 


Percentage of students testing below proficient, 2001/02 


151 


31 


19 


739 


28 


16 


Percentage of students testing proficient, 2001/02 


151 


56 


14 


739 


56 


11 


Percentage of students testing advanced, 2001/02 


151 


14 


11 


739 


16 


12 


Change in percentage testing below proficient, 2001/02- 
2004/05 


151 


-11 


19 


739 


-10 


13 


Change in percentage testing proficient, 2001/02-2004/05 


151 


0 


16 


739 


0 


12 


Change in percentage testing advanced, 2001/02-2004/05 


151 


10 


12 


739 


10 


10 



Source: Authors' calculations based on data from Virginia Department of Education (2007) and National Center for Education Statistics (2007). 



for grades 3 and 5 and 95 percent of schools that 
reported math assessments for grades 3 and 5. 

The sample would not have increased much for 
the grade 8 assessments, but would have increased 
to 98 percent of schools for end of course reading 
and 95 percent for Algebra 2. Table A5 displays the 
number of schools in each category. 



excluded schools with 10-49 test-takers each year 
and the sample schools. Descriptive statistics are 
not presented for schools with 9 or fewer test- 
takers in any year 2001/02-2004/05 because the 
state did not report the data. 



Methodology for the bivariate analysis 



As in Kentucky, the schools excluded in Virginia 
are smaller (based on total enrollments in 2002) 
than the sample schools and have larger percent- 
ages of students eligible for free or reduced-price 
lunch. Tables A6, A7, and A8 provide details on 
the school characteristics and key variables for 



The bivariate analysis examines, in four steps, the 
relationship between the annualized changes in 
the school-level percentage of students passing 
and annualized changes in school-level percentage 
testing advanced over five years in Kentucky and 
four years in Virginia. 
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TABLE A7 

Comparison of excluded and sample middle schools in Virginia, 2001/02-2004/05 





Excluded schools 
(10 49 test takers in any year) 


Sample schools (50 or more 
test takers in each year) 


Grade, subject, and characteristic 


Number 
of schools 


Mean 


Standard 

deviation 


Number 
of schools 


Mean 


Standard 

deviation 


Grade 8 reading 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


3 


36 


11 


323 


32 


19 


Percentage of racial/ethnic minority students, 2001/02 (school) 


3 


11 


5 


323 


34 


26 


Total students at school, 2001/02 


3 


283 


58 


323 


817 


446 


Percentage of students testing below proficient, 2001/02 


3 


30 


12 


323 


31 


12 


Percentage of students testing proficient, 2001/02 


3 


59 


10 


323 


51 


7 


Percentage of students testing advanced, 2001/02 


3 


11 


3 


323 


18 


9 


Change in percentage testing below proficient, 2001/02- 
2004/05 


3 


-1 


11 


323 


-7 


8 


Change in percentage testing proficient, 2001/02-2004/05 


3 


-8 


12 


323 


2 


9 


Change in percentage testing advanced, 2001/02-2004/05 


3 


8 


6 


323 


5 


7 


Grade 8 math 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


9 


40 


18 


321 


32 


19 


Percentage of racial/ethnic minority students, 2001/02 (school) 


9 


25 


24 


321 


34 


26 


Total Students at School, 2001/02 


9 


385 


178 


321 


819 


445 


Percentage of students testing below proficient, 2001/02 


9 


26 


23 


321 


30 


16 


Percentage of students testing proficient, 2001/02 


9 


63 


20 


321 


53 


10 


Percentage of students testing advanced, 2001/02 


9 


12 


13 


321 


17 


12 


Change in percentage testing below proficient, 2001/02-2004/05 


9 


-1 


13 


321 


-11 


11 


Change in percentage testing proficient, 2001/02-2004/05 


9 


2 


15 


321 


8 


11 


Change in percentage testing advanced, 2001/02-2004/05 


9 


-2 


8 


321 


3 


7 



Source: Authors' calculations based on data from Virginia Department of Education (2007) and National Center for Education Statistics (2007). 



First, the average annualized change in school-level 
percentage of students testing advanced is compared 
across four groups of schools with increasingly 
greater changes in the percentage of students passing: 
schools where the change in percentage passing is 
more than one standard deviation below the mean, 
schools where it is within one standard deviation 
below the mean, schools where it is within one stan- 
dard deviation above the mean, and schools where it 
is more than one standard deviation above the mean. 

Second, the bivariate association is examined 
using scatterplots to display annualized changes 
in the school-level percentage of students passing 
by annualized changes in school-level percentage 
testing advanced. 



Third, an ordinary least squares (OLS) regres- 
sion is estimated and plotted with a 95 percent 
confidence interval. The model’s dependent 
variable, the annualized change in the percentage 
of students testing advanced, is a function of the 
annualized change in the percentage passing, the 
independent variable, or, 

Kentucky: A adv {ti _ t(j)l4 = a + (3JA pass {k _ m ] + e 

Virginia: A adv (h _ k)li = a + PJApass^/j] + e 

where: A adv = change in the percentage testing 
advanced, t 0 = 2001/02, t l = 2002/03, f 3 = 2004/05, 
f 4 = 2005/06, and A pass = change in the percent- 
age passing. 




20 



PERCENTAGE OF STUDENTS PASSING & PERCENTAGE TESTING ADVANCED ON KENTUCKY & VIRGINIA ASSESSMENTS 



TABLE A8 

Comparison of excluded and sample high schools in Virginia, 2001/02- 


-2004/05 










Excluded schools 
(10 49 test takers in any year) 


Sample schools (50 or more 
test takers in each year) 


Grade, subject, and characteristics 


Number 
of schools 


Mean 


Standard 

deviation 


Number 
of schools 


Mean 


Standard 

deviation 


Reading (end of course exam) 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


16 


25 


20 


279 


22 


17 


Percentage of racial/ethnic minority students, 2001/02 
(school) 


16 


33 


37 


279 


31 


24 


Total students at school, 2001/02 


16 


305 


85 


279 


1,228 


665 


Percentage of students testing below proficient, 2001/02 


16 


20 


18 


279 


16 


8 


Percentage of students testing proficient, 2001/02 


16 


61 


14 


279 


59 


8 


Percentage of students testing advanced, 2001/02 


16 


18 


17 


279 


25 


11 


Change in percentage testing below proficient, 2001/02- 
2004/05 


16 


-8 


11 


279 


-4 


8 


Change in percentage testing proficient, 2001/02-2004/05 


16 


7 


13 


279 


3 


7 


Change in percentage testing advanced, 2001/02-2004/05 


16 


2 


11 


279 


2 


7 


Algebra 2 (end of course exam) 


Percentage of students eligible for free or reduced-price 
lunch, 2001/02 (school) 


41 


39 


22 


232 


19 


14 


Percentage of racial/ethnic minority students, 2001/02 
(school) 


41 


28 


31 


232 


32 


23 


Total students at school, 2001/02 


41 


482 


218 


232 


1,346 


639 


Percentage of students testing below proficient, 2001/02 


41 


27 


18 


232 


24 


14 


Percentage of students testing proficient, 2001/02 


41 


56 


12 


232 


54 


10 


Percentage of students testing advanced, 2001/02 


41 


17 


15 


232 


22 


12 


Change in percentage testing below proficient, 2001/02- 
2004/05 


41 


-18 


20 


232 


-13 


12 


Change in percentage testing proficient, 2001/02-2004/05 


41 


7 


19 


232 


2 


12 


Change in percentage testing advanced, 2001/02-2004/05 


41 


11 


17 


232 


10 


11 



Source: Authors' calculations based on data from Virginia Department of Education (2007) and National Center for Education Statistics (2007). 



The analysis compares the slope coefficients and 
the r 2 values to test the direction, magnitude, and 
strength of any relationships. 

Fourth, the sample was split into two groups: 
schools with the percentage of students passing 
in 2001/02 below the 2004/05 annual measurable 
objective and schools with the percentage passing 
above the 2004/05 objective. Splitting the sample 
in this way enables testing for whether schools 
with different levels of proficiency in 2001/02 show 
the same pattern. 



Methodology for the multivariate analysis 

Three separate OLS regressions were conducted 
in which the dependent variable is the annualized 
change in the school-level percentage of students 
testing advanced (over 2002/03-2005/06 in Ken- 
tucky and 2002/03-2004/05 in Virginia) and the 
independent variable is the annualized change in 
the school-level percentage passing. This change 
was made so that the percentage testing proficient 
in 2001/02 could be used as a control. The mod- 
els were estimated controlling for no covariates, 
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controlling for the percentage of students testing 
proficient in 2001/02, and controlling for school- 
level characteristics (percentage of students 
eligible for free or reduced-price lunch [FRL] and 
changes in that percentage, percentage of racial/ 
ethnic minority students and changes in that 
percentage, and changes in school locale — city, 
suburb, town, or rural area): 

(la) Kentucky: A adv (k _ h)l3 = a + (3j[A pass (ti _ m \ + e 

(lb) Kentucky: A idv ik _ h)l3 = a + ft [A pass {k _ t ^\ + 
(3 2 (% proficient , o ) + e 

(lc) Kentucky: A adv {u _ m = a + (3 1 [Apass ((4 _ fi)/3 ] 

+ (3 2 (% proficient^ ) + (3 3 (% FRL eligible t ) + 
(3 4 [A % FRL( ti _ t( fi + (3 5 (% racial/ethnic minori- 
ty , , o ) + (3 6 (A % racial/ethnic minority ( , 4 _ (o) ) + 
H cit yt) + ^s(suburb to ) + (3 9 (town, o ) + e 

(2a) Virginia: A adv (t} _ m = a + ^[Apass {h _ ti)n ] + e 

(2b) Virginia: A adv {h _ m = a + (3JA pass {h _ tiV2 ] + 

(3 2 (% proficient to ) + e 

(2c) Virginia: A adv (h _ h)l2 = a + (3JA pass {h _ h)ll ] + 

(3 2 (% proficient tg ) + |3 3 (% FRL eligible fo ) + 

(3 4 [A % FRL^q] + (3 S (% racial/ethnic mino- 
rity t J + (3 6 [A % racial/ethnic minority ( f ( )] + 
(3 7 (cffj)/ fo ) + (3 s (suburb t ) + (3 9 {town t ) + e 

where A adv = change in the percentage testing 
advanced, t 0 = 2001/02, t x - 2002/03, f 3 = 2004/05, 
f 4 = 2005/06, and A pass = change in the percent- 
age passing. 

The annualized change in the school-level percent- 
age of students testing advanced is assumed to 
be linearly related to the independent variables, 
defined as follows: 

• % proficient (fl is the percentage of students at 

the school in the grade tested testing at the 
proficient level in 2001/02, as reported by the 
Kentucky Department of Education (2007) 
or the Virginia Department of Education 
(2007). 



• % FRL eligible tg is the percentage of students 
at the school eligible for free or reduced-price 
lunch in 2001/02, as reported in the Common 
Core of Data for 2001/02 (U.S. Department of 
Education 2007). 

• A % FRL eligible t4 _ tg is the difference in the 
percentage of students at the school eligible 
for free or reduced-price lunch in 2005/06 
and 2001/02, as reported in the Common 
Core of Data for 2001/02 and for 2005/06 (U.S. 
Department of Education 2007). 

• % racial/ethnic minority t is the percentage of 
racial/ethnic minority students at the school, 
as reported in the Common Core of Data for 
2001/02 (U.S. Department of Education 2007). 

• A % racial/ethnic minority t t is the difference 
in the percentage of students of a racial/ethnic 
minority at the school in 2005/06 and 
2001/02, as reported in the Common Core 

of Data for 2001/02 and for 2005/06 (U.S. 
Department of Education 2007). 

• City t is a zero-one indicator, equal to one if a 
school has urban-centric locale codes 11, 12, or 
13, as reported in the Common Core of Data for 
2001/02 (U.S. Department of Education 2007). 

• Suburb t is, a zero-one indicator, equal to one 
if a school has urban-centric locale codes 21, 
22, or 23, as reported in the Common Core of 
Data for 2001/02 (U.S. Department of Educa- 
tion 2007). 

• Town t is a zero-one indicator, equal to one 
if a school has urban-centric locale codes 31, 
32, or 33, as reported in the Common Core of 
Data for 2001/02 (U.S. Department of Educa- 
tion 2007). 

The slope coefficients measure the degree of as- 
sociation between the dependent and independent 
variables, holding constant other factors included 
in the model. (The coefficients for each covariate 
are reported in appendix C.) 
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APPENDIX B 

DETAILED RESULTS OF THE BIVARIATE ANALYSIS 

The report analyzes annualized changes at the 
school level in the percentage of students testing 
advanced as a function of annualized changes 
in the percentage passing (testing proficient 
or advanced) in reading and math by grade in 
Kentucky and Virginia. The bivariate results 
examine changes in Kentucky over the five years 
2001/02 -2005/06 and in Virginia over the four 
years 2001/02-2004/05. Descriptive statistics for 
these annualized changes are presented in tables 
B1 and B2. 



TABLE B1 

Descriptive statistics for bivariate analysis of the association between annualized changes in the percentage 
of students testing advanced and in the percentage passing in Kentucky, 2001/02-2005/06 



Subject, grade, and variable 


Number of 
schools 


Mean 


Standard 

deviation 


Reading 


Grade 4 


Annualized change in percentage passing 


616 


2.39 


3.52 


Annualized change in percentage testing advanced 


616 


1.31 


2.40 


Grade 7 


Annualized change in percentage passing 


202 


1.72 


2.72 


Annualized change in percentage testing advanced 


202 


0.37 


1.40 


Grade 10 


Annualized change in percentage passing 


197 


3.15 


2.26 


Annualized change in percentage testing advanced 


197 


2.18 


1.67 


Math 


Grade 5 


Annualized change in percentage passing 


604 


5.24 


3.75 


Annualized change in percentage testing advanced 


604 


2.69 


2.36 


Grade 8 


Annualized change in percentage passing 


202 


2.16 


2.30 


Annualized change in percentage testing advanced 


202 


0.97 


1.13 


Grade 11 


Annualized change in percentage passing 


197 


2.05 


2.01 


Annualized change in percentage testing advanced 


197 


1.19 


1.22 



Source: Authors' calculations based on data from Kentucky Department of Education (2007). 
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TABLE B2 

Descriptive statistics for bivariate analysis of association between annualized changes in the percentage of 
students testing advanced and in the percentage passing in Virginia, 2001/002-2004/05 



Subject, grade, and variable 


Number of 
schools 


Mean 


Standard 

deviation 


Reading 


Grade 3 


Annualized change in percentage passing 


710 


1.59 


4.00 


Annualized change in percentage testing advanced 


710 


0.55 


3.07 


Grade 5 


Annualized change in percentage passing 


737 


2.57 


3.15 


Annualized change in percentage testing advanced 


737 


4.02 


2.87 


Grade 8 


Annualized change in percentage passing 


323 


2.28 


2.57 


Annualized change in percentage testing advanced 


323 


4.02 


2.38 


High school reading (end of course exam) 


Annualized change in percentage passing 


279 


1.43 


2.51 


Annualized change in percentage testing advanced 


279 


0.78 


3.75 


Math 


Grade 3 


Annualized change in percentage passing 


723 


2.40 


3.31 


Annualized change in percentage testing advanced 


723 


4.02 


4.53 


Grade 5 


Annualized change in percentage passing 


739 


3.31 


4.41 


Annualized change in percentage testing advanced 


739 


3.28 


3.40 


Grade 8 


Annualized change in percentage passing 


321 


3.60 


3.75 


Annualized change in percentage testing advanced 


321 


0.84 


2.39 


High school Algebra 2 (end of course exam) 


Annualized change in percentage passing 


232 


4.16 


3.92 


Annualized change in percentage testing advanced 


232 


3.43 


3.59 



Source: Authors' calculations based on data from Virginia Department of Education (2007). 



Average changes in the school-level percentage 
of students testing advanced for schools grouped 
by the deviation from the mean change in the 
percentage of students passing for Kentucky 

The average annualized changes in the percent- 
age of students testing advanced was analyzed by 
schools grouped by standard deviation based on 
annualized changes in the percentage of students 
passing. The resulting figures show that schools 
with the largest changes in the percentage of 



students passing (those with changes more than 
one standard deviation above the mean) also 
have the highest average annualized changes in 
the percentage testing advanced. This positive 
relationship holds whether schools are grouped 
by standard deviations from the mean or percent- 
age point changes from the mean. Figures B1-B6 
display these changes for schools in Kentucky. See 
table B1 for the means and standard deviations 
used to set cutpoints for each group, by grade and 
by subject. 
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FIGURE B1 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 4 reading in 
Kentucky schools grouped by standard deviation, 
2001/02-2005/06 

Annualized change in percentage testing advanced 
6 



FIGURE B2 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 7 reading in 
Kentucky schools grouped by standard deviation, 
2001/02-2005/06 

Annualized change in percentage testing advanced 
6 
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Source: Authors' calculations based on data from Kentucky Department 
of Education (2007). 



Source: Authors' calculations based on data from Kentucky Department 
of Education (2007). 



FIGURE B3 

Association between annualized changes in the 
percentage of students testing advanced and in 
the percentage passing in grade 10 reading in 
Kentucky schools grouped by standard deviation, 
2001/02-2005/06 

Annualized change in percentage testing advanced 
6 



4.15 




FIGURE B4 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 5 math in 
Kentucky schools grouped by standard deviation, 
2001/02-2005/06 

Annualized change in percentage testing advanced 




-2 

More than 1 Within 1 

standard deviation standard deviation 
below the mean below the mean 
(n = 28) (n = 64) 



Within 1 More than 1 

standard deviation standard deviation 
above the mean above the mean 
(n = 72) (n = 33) 



More than 1 Within 1 Within 1 More than 1 

standard deviation standard deviation standard deviation standard deviation 
below the mean below the mean above the mean above the mean 
(n = 96) (n = 210) (n = 200) (n = 98) 
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Source: Authors' calculations based on data from Kentucky Department 
of Education (2007). 



Source: Authors' calculations based on data from Kentucky Department 
of Education (2007). 
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FIGURE B5 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 7 math in 
Kentucky schools grouped by standard deviation, 
2001/02-2005/06 

Annualized change in percentage testing advanced 
6 



FIGURE B6 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 11 math in 
Kentucky schools grouped by standard deviation, 
2001/02-2005/06 

Annualized change in percentage testing advanced 
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Source: Authors' calculations based on data from Kentucky Department 
of Education (2007). 



Source: Authors' calculations based on data from Kentucky Department 
of Education (2007). 



Regression results for the association between annualized 
changes in the percentage of students testing advanced 
and in the percentage passing in Kentucky schools 

Scatterplot diagrams illustrate the regression results 
for the association between annualized school- 
level changes in the percentage of students testing 
advanced and annualized changes in the percentage 
passing for reading and math in Kentucky schools 
(figures B7 and B8). Each scatterplot includes a lin- 
ear regression line and lines designating a 95 percent 
confidence interval. In each case the trend is posi- 
tive, meaning that schools with positive changes in 
the percentage of students passing also have positive 
changes in the percentage testing advanced. 



Each scatterplot can be divided into four quad- 
rants: positive changes in percentage of students 
passing and percentage testing advanced (top 
right); positive change in percentage of students 
passing but negative change in percentage testing 
advanced (bottom right); negative change in per- 
centage of students passing and percentage test- 
ing advanced (bottom left); and negative change 
in percentage of students passing but positive 
change in percentage testing advanced (top left). 
Table B3 shows the percentage of schools in each 
quadrant. Table B4 displays the corresponding 
regression coefficients, correlation coefficients, 
and r 2 values. 
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FIGURE B7 

Regression results for the association between annualized changes in the percentage of students testing 
advanced and in the percentage passing in reading in Kentucky schools, 2001/02-2005/06 





Annualized change in percentage passing 



Source: Authors' calculations based on data from Kentucky Department of Education (2007). 



TABLE B3 

Percentage of Kentucky schools in each quadrant of figures B7 and B8 by subject and grade, 2001/02-2005/06 



Subject and grade 


Positive passing, 
positive advanced 


Positive passing, 
negative advanced 


Negative passing, 
negative advanced 


Negative passing, 
positive advanced 


Reading 


Grade 4 


67.7 


11.0 


10.7 


10.6 


Grade 7 


58.4 


19.3 


15.3 


6.9 


Grade 10 


87.8 


4.1 


2.5 


5.6 


Math 


Grade 5 


87.9 


4.8 


2.8 


4.5 


Grade 8 


81.2 


6.9 


2.5 


9.4 


Grade 11 


80.2 


4.6 


6.6 


8.6 



Source: Authors' calculations based on data from Kentucky Department of Education (2007). 
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FIGURE B8 

Regression results for the association between the annualized changes in the percentage of students testing 
advanced and in the percentage passing in math in Kentucky schools, 2001/02-2005/06 






Source: Authors' calculations based on data from Kentucky Department of Education (2007). 



TABLE B4 

Ordinary least squares regressions for the association between changes in the percentage of students 
testing advanced and in the percentage passing in Kentucky by subject and grade, 2001/02-2005/06 



Subject and grade 


Number of schools 


Intercept 

coefficient 


Slope coefficient 


r 2 


Correlation 

coefficient 


Reading 


Grade 4 


616 


0.640*** 


0.282*** 


.170 


0.412 


Grade 7 


202 


-0.186* 


0.324*** 


.399 


0.632 


Grade 10 


197 


0.536*** 


0.524*** 


.502 


0.709 


Math 


Grade 5 


604 


0.824*** 


0.352*** 


.312 


0.559 


Grade 8 


202 


0.214** 


0.350*** 


.510 


0.714 


Grade 11 


197 


0.368*** 


0.400*** 


.434 


0.659 



* Significant at p < .05; ^^significant at p < .01; ^““significant at p < .001 . 

Source: Authors' calculations based on data from Kentucky Department of Education (2007). 
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Average changes in the school-level percentage 
of students testing advanced for schools grouped 
by the deviation from the mean change in the 
percentage of students passing for Virginia 

Schools with the largest changes in the percent- 
age of students passing (those with changes more 
than one standard deviation above the mean) 
have a higher annualized change in the percent- 
age testing advanced than schools with the lowest 
changes in the percentage passing (those more 
than one standard deviation below the mean, 
figures B9-B16). 



FIGURE B9 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 3 reading in 
Virginia schools grouped by standard deviation, 
2001/02-2004/05 

Annualized change in percentage testing advanced 
10 



FIGURE BIO 

Association between annualized changes in the 
percentage of student testing advanced and in 
the percentage passing in grade 5 reading in 
Virginia schools grouped by standard deviation, 
2001/02-2004/05 

Annualized change in percentage testing advanced 
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Source: Authors' calculations based on data from Virginia Department 
of Education (2007). 



Annualized change in percentage passing 

Source: Authors' calculations based on data from Virginia Department 
of Education (2007). 
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FIGURE B11 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 8 reading in 
Virginia schools grouped by standard deviation, 
2001/02-2004/05 

Annualized change in percentage testing advanced 
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Source: Authors' calculations based on data from Virginia Department 
of Education (2007). 



FIGURE B12 

Association between annualized changes in the 
percentage of students testing advanced and in 
the percentage passing in high school reading 
(end of course exams) in Virginia schools grouped 
by standard deviation, 2001/02-2004/05 
Annualized change in percentage testing advanced 
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Source: Authors' calculations based on data from Virginia Department 
of Education (2007). 



FIGURE B13 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 3 math in 
Virginia schools grouped by standard deviation, 
2001/02-2004/05 



FIGURE B14 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 5 math in 
Virginia schools grouped by standard deviation, 
2001/02-2004/05 
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Source: Authors' calculations based on data from Virginia Department 
of Education (2007). 



Source: Authors' calculations based on data from Virginia Department 
of Education (2007). 
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FIGURE B15 

Association between annualized changes in the 
percentage of students testing advanced and 
in the percentage passing in grade 8 math in 
Virginia schools grouped by standard deviation, 
2001/02-2004/05 

Annualized change in percentage testing advanced 
10 

8 



FIGURE B16 

Association between annualized changes in the 
percentage of students testing advanced and in 
the percentage passing in high school Algebra 2 
(end of course exam) in Virginia schools grouped 
by standard deviation, 2001/02-2004/05 
Annualized change in percentage testing advanced 
10 

8 



6 



4 



2 



0 




_2 - 1.28 





2.30 





-4 -4 



More than 1 


Within 1 


Within 1 


More than 1 


More than 1 


Within 1 


Within 1 


More than 1 


standard deviation 


standard deviation 


standard deviation 


standard deviation 


standard deviation 


standard deviation 


standard deviation 


standard deviation 


below the mean 


below the mean 


above the mean 


above the mean 


below the mean 


below the mean 


above the mean 


above the mean 


(n = 38) 


(n = 139) 


(n = 105) 


(n = 39) 


(n = 22) 


(n = 110) 


(n = 68) 


(n = 32) 



Annualized change in percentage passing Annualized change in percentage passing 
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of Education (2007). of Education (2007). 



Regression results for the association between annualized 
changes in the percentage of students testing advanced 
and in the percentage passing in Virginia schools 

Scatterplot diagrams illustrate the regression 
results for the association between annualized 
school-level changes in the percentage of students 
testing advanced and the annualized changes in 
the percentage passing for reading and math in 
Virginia schools (figures B17 and B18). 

Each scatterplot can be divided into four quad- 
rants: positive changes in percentage of students 



passing and percentage testing advanced (top 
right); positive change in percentage of students 
passing but negative change in percentage testing 
advanced (bottom right); negative change in per- 
centage of students passing and percentage test- 
ing advanced (bottom left); and negative change 
in percentage of students passing but positive 
change in percentage testing advanced (top left). 
Table B5 shows the percentage of schools in each 
quadrant. Table B6 displays the corresponding 
regression coefficients, correlation coefficients, 
and r 2 values. 
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FIGURE B17 

Regression results for the association between annualized changes in the percentage of students testing 
advanced and in the percentage passing in reading in Virginia schools, 2001/02-2004/05 
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Source: Authors' calculations based on data from Virginia Department of Education (2007). 



TABLE B5 

Percentage of Virginia schools in each quadrant of figures B17 and B18 by subject and grade 



Subject and grade 


Positive passing, 
positive advanced 


Positive passing, 
negative advanced 


Negative passing, 
negative advanced 


Negative passing, 
positive advanced 


Reading 


Grade 3 


47.7 


16.8 


22.5 


13.0 


Grade 5 


53.1 


31.6 


7.2 


8.2 


Grade 8 


72.1 


16.3 


7.1 


4.6 


High school reading (end of course exam) 


52.4 


20.0 


18.0 


9.6 


Math 


Grade 3 


73.9 


8.3 


9.4 


8.4 


Grade 5 


71.7 


7.0 


7.4 


13.8 


Grade 8 


60.7 


27.4 


8.7 


3.1 


High school Algebra 2 (end of course exam) 


84.5 


7.3 


4.7 


3.4 



Source: Authors' calculations based on data from Virginia Department of Education (2007). 
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FIGURE B18 

Regression results for the association between annualized changes in the percentage of students testing 
advanced and in the percentage passing in math in Virginia schools, 2001/02-2004/05 
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Source: Authors' calculations based on data from Virginia Department of Education (2007). 
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TABLE B6 

Ordinary least squares regressions for the association between changes in the percentage of students 
testing advanced and in the percentage passing in Virginia by subject and grade, 2001/02-2004/05 



Subject and grade 


Number of 
schools 


Intercept 

coefficient 


Slope 

coefficient 


r 2 


Correlation 

coefficient 


Reading 


Grade 3 


710 


-0.186 


0.404*** 


.280 


0.529 


Grade 5 


737 


3.520“* 


0.162*** 


.035 


0.187 


Grade 8 


323 


0.874*“ 


0.323*“ 


.109 


0.330 


High school reading (end of course exam) 


270 


-0.383 


0.793“* 


.273 


0.522 


Math 


Grade 3 


723 


1.894*“ 


0.887*“ 


.420 


0.648 


Grade 5 


739 


2.076*** 


0.364*** 


.223 


0.472 


Grade 8 


321 


-0.094 


0.260*** 


.167 


0.409 


High school Algebra 2 (end of course exam) 


232 


1.868*“ 


0.374*** 


.168 


0.410 



“* Significant at p < .001. 

Source: Authors' calculations based on data from Virginia Department of Education (2007). 



TABLE B7 

Ordinary least squares regressions for the association between changes in the percentage of students 
testing advanced and in the percentage passing for schools whose percentage passing in 2001/02 was below 
the 2004/05 annual measurable objective in Kentucky (2001/02-2005/06) and Virginia (2001/02-2004/05) 



State, subject, and grade 


Number of 
schools 


Intercept 

coefficient 


Slope 

coefficient 


r 2 


Correlation 

coefficient 


Kentucky 


Reading 


Grade 4 


45 


-0.186 


0.208*** 


.344 


.587** 


Grade 7 


25 


-0.272 


0.235“ 


.300 


.548“ 


Grade 10 


55 


0.126 


0.429“* 


.577 


.760** 


Math 


Grade 5 


65 


-0.206 


0.346*“ 


.426 


.680** 


Grade 8 


55 


0.152 


0.235*“ 


.344 


.586** 


Grade 11 


49 


0.573* 


0.379*“ 


.494 


.703** 


Virginia 


Reading 


Grade 3 


271 


0.274 


0.312*** 


.233 


.483** 


Grade 5 


160 


1.594“* 


0.290*** 


.151 


.389“ 


Grade 8 


167 


0.861*** 


0.229“* 


.100 


.316“ 


High school reading (end of course exam) 


12 


0.750 


0.445 


.086 


.293 


Math 


Grade 3 


114 


1.603* 


0.806*“ 


.452 


.672** 


Grade 5 


298 


1.452*“ 


0.366*** 


.275 


.524** 


Grade 8 


145 


-0.355 


0.235“* 


.231 


.480“ 


High school Algebra 2 (end of course exam) 


59 


1.424 


0.295* 


.108 


.328“ 



‘Significant at p < .05; “significant at p < .01; “‘significant at p < .001 . 

Source: Authors' calculations based on data from Kentucky Department of Education (2007) and Virginia Department of Education (2007). 
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Detailed analysis of schools with the percentage 
of students passing in 2001/02 below or above 
the 2004/05 annual measurable objective 

The NCLB Act sanctions schools only if insufficient 
numbers of students fail to test at the proficient 
level or above on standardized tests. Thus the study 
considered schools where the 2001/02 percentage 
of students passing was below the 2004/05 annual 
measurable objective as having the strongest incen- 
tives to shift resources to below proficient students. 
Those schools needed to increase the percentage of 
students testing proficient to meet the annual mea- 
surable objective in 2004/05, the year the objective 
was increased for the first time. 



To test whether schools with the percentage of 
student passing in 2001/02 below the 2004/05 
annual measurable objective display the same 
trends as the sample as a whole, the sample was 
split into two groups: schools with the percentage 
of students passing in 2001/02 below the 2004/05 
annual measurable objective and those with the 
percentage passing in 2001/02 above the 2004/05 
objective. 

In Kentucky the annual measurable objectives 
varied by subject and school level (for elementary 
schools, it was 53.86 in reading and 32.14 in math; 
for middle schools it was 52.40 in reading and 
26.93 in math; for high schools it was 29.35 in 



TABLE B8 

Ordinary least squares regressions for the association between changes in the percentage of students 
testing advanced and in the percentage passing for schools whose percentage passing in 2001/02 was above 
the 2004/05 annual measurable objective in Kentucky (2001/02-2005/06) and Virginia (2001/02-2004/05) 



State, subject, and grade 


Number of 
schools 


Intercept 

coefficient 


Slope 

coefficient 


r 2 


Correlation 

coefficient 


Kentucky 


Reading 


Grade 4 


571 


0.645*** 


0.327*** 


.192 


.438** 


Grade 7 


177 


-0.160 


0.343*** 


.417 


.646** 


Grade 10 


142 


0.545*** 


0.633*** 


.610 


.781** 


Math 


Grade 5 


539 


0.842*** 


0.378*** 


.328 


.573** 


Grade 8 


147 


0.279*** 


0.388*** 


.584 


.764** 


Grade 11 


148 


0.532** 


0.573*** 


.536 


.732** 


Virginia 


Reading 


Grade 3 


439 


-0.005 


0.537*** 


.199 


.446** 


Grade 5 


577 


3.574*** 


0.396*** 


.097 


.311* 


Grade 8 


156 


0.778*** 


0.725*** 


.234 


.484** 


High school reading (end of course exam) 


258 


-0.333 


0.826*** 


.240 


.490** 


Math 


Grade 3 


609 


1.786*** 


1.062*** 


.362 


.602** 


Grade 5 


441 


2.163*** 


0.658*** 


.261 


.511** 


Grade 8 


176 


-0.413 


0.649*** 


.262 


.512** 


High school Algebra 2 (end of course exam) 


173 


1.063** 


0.838*** 


.320 


.566** 



* Significant at p < .05;** significant at p < .01;*** significant at p < .001. 

Source: Authors' calculations based on data from Kentucky Department of Education (2007) and Virginia Department of Education (2007). 
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reading and 29.79 in math). Virginia had the same 
original annual measurable objective for both 
reading and math in 2004/05 (70). (Virginia later 
revised its annual measurable objectives to 65 in 
reading and 63 in math.) 

The regression results for the association between 
annualized changes in the percentage of students 
passing and annualized changes in the percent- 
age testing advanced for Kentucky and Virginia 
schools with percentage passing rates in 2001/02 
below the 2004/05 annual measurable objective 
and those with percentage passing rates in 2001/02 
above the 2004/05 annual measurable objective 
are shown in tables B7 and B8. In nearly all cases 
the slope (which tells the direction and magnitude 
of the association) is positive and statistically 
significant at the .05 level, indicating that changes 
in the percentage of students passing are positively 
associated with the changes in the percentage of 
students testing advanced. Among the schools 
with the percentage passing in 2001/02 below the 
2004/05 annual measurable objective, the one 
exception is the Virginia high school reading (end 
of course exams), but this could be because only 
12 schools were in the “below” category. The slope 
coefficients of the schools in the “above” category 



are larger than those of the schools in the “below” 
category. This means that in schools where the 
percentage of students passing in 2001/02 was 
higher than the 2004/05 annual measurable objec- 
tive, the change in percentage of students testing 
advanced made up a greater proportion of the 
increase in the percentage passing than it did in 
schools where the percentage passing in 2001/02 
was below the 2004/05 annual measurable objec- 
tive. For both groups, the association was positive. 

In both Kentucky and Virginia the r 2 values (a 
measure of the model’s goodness of fit or ex- 
plained variance) of the regressions for the schools 
in the “below” category are greater than those of 
the schools in the “above” category for elemen- 
tary schools but lower than those for the schools 
in the “above” category for middle and high 
schools. These results suggest that for schools in 
the “below” category, the relationship between 
changes in the percentage of students passing and 
changes in the percentage testing advanced is posi- 
tive, but less steep than for schools in the “above” 
category. Also noteworthy is that the model fits 
the Kentucky data better than it does the Virginia 
data — the Kentucky r 2 values average about twice 
those of Virginia. 
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APPENDIX C 

DETAILED RESULTS OF THE 

MULTIVARIATE ANALYSIS 

Analyzing the association between changes in 
the percentage of students testing advanced as a 
function of the changes in the percentage test- 
ing proficient or above while controlling for the 
percentage testing proficient in 2001/02 and other 
school characteristics required switching to a 
four-year period in Kentucky and a three-year 
period in Virginia. The models are specified so 
that the percentage testing proficient in 2001/02 is 
not included among the independent or dependent 
variables. Descriptive statistics for reading and for 
math in Kentucky are in tables Cl and C2. 

The results of models la, lb, and lc (specified in ap- 
pendix A, “Methodology for the multivariate analy- 
sis”) are presented in tables C3 and C4. In all cases 
the association between changes in the percentage 
of students passing and changes in the percentage 
testing advanced is statistically significant (model 
la). The positive and statistically significant slope 
coefficients suggest that schools with the largest 
increases in the percentage of students passing also 
show the largest increases in the percentage testing 
advanced, and also that those with the smallest 
increases in the percentage of students passing 
also show the smallest increases in the percentage 



testing advanced. When controls for the percentage 
of students testing proficient in 2001/02 are added 
(model lb), the association holds and the amount 
of variance explained. Model lc adds when school- 
level characteristics are added (model lc), and the 
association continues to hold. 

Descriptive statistics for reading and math in 
Virginia are in tables C5 and table C6. 

The results for models 2a, 2b, and 2c (as specified 
in appendix A) are presented in tables C7 and C8. 
The results are weaker for Virginia than for Ken- 
tucky (with the exception of grade 3 reading and 
math), as shown by the R 2 values, but the statisti- 
cally significant positive relationship between the 
changes in the percentage of students passing and 
changes in the percentage testing advanced holds. 
Across the models the association holds regard- 
less of controls added. The percentage proficient 
in 2001/02 and school-level variables reinforce the 
relationship. The amount of variance explained 
increases with each model, but the underlying 
relationship between the key independent variable 
and changes in the percentage testing advanced 
remains the same. 

The results of models 2a, 2b, and 2c (specified in 
appendix A, “Methodology for the multivariate 
analysis”) are presented in tables C7 and C8. 
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TABLE Cl 

Descriptive statistics for multivariate analysis for reading in Kentucky, 2002/03-2005/06 



Grade and variable 


Number of 
schools 


Mean 


Standard 

deviation 


Grade 4 


Change in percentage testing advanced, 2002/03-2005/06 


591 


1.57 


3.17 


Change in percentage passing, 2002/03-2005/06 


591 


2.54 


4.30 


Percentage testing proficient, 2001/02 


591 


54.38 


12.77 


Percentage of students eligible for free or reduced-price lunch 


591 


59.07 


23.02 


Change in percentage of students eligible for free or reduced-price lunch 


591 


3.27 


13.31 


Percentage of racial/ethnic minority students 


591 


13.07 


17.13 


Change in percentage of racial/ethnic minority students 


591 


1.41 


3.88 


City 


591 


0.21 


0.41 


Suburb 


591 


0.13 


0.34 


Town 


591 


0.18 


0.38 


Rural (comparison) 


591 


0.48 


0.50 


Grade 7 


Change in percentage testing advanced, 2002/03-2005/06 


194 


0.12 


1.74 


Change in percentage passing, 2002/03-2005/06 


194 


1.89 


3.33 


Percentage testing proficient, 2001/02 


194 


49.84 


9.84 


Percentage of students eligible for free or reduced-price lunch 


194 


47.51 


19.40 


Change in percentage of students eligible for free or reduced-price lunch 


194 


4.60 


11.69 


Percentage of racial/ethnic minority students 


194 


11.22 


14.13 


Change in percentage of racial/ethnic minority students 


194 


1.57 


3.98 


City 


194 


0.19 


0.39 


Suburb 


194 


0.12 


0.32 


Town 


194 


0.29 


0.46 


Rural (comparison) 


194 


0.40 


0.49 


Grade 10 


Change in percentage testing advanced, 2002/03-2005/06 


194 


2.53 


1.97 


Change in percentage passing, 2002/03-2005/06 


194 


3.31 


2.56 


Percentage testing proficient, 2001/02 


194 


21.01 


7.06 


Percentage of students eligible for free or reduced-price lunch 


194 


37.89 


18.00 


Change in percentage of students eligible for free or reduced-price lunch 


194 


4.75 


10.44 


Percentage of racial/ethnic minority students 


194 


9.43 


12.40 


Change in percentage of racial/ethnic minority students 


194 


1.40 


2.94 


City 


194 


0.13 


0.34 


Suburb 


194 


0.09 


0.29 


Town 


194 


0.29 


0.45 


Rural (comparison) 


194 


0.48 


0.50 



Source: Authors' calculations based on data from Kentucky Department of Education (2007); U.S. Department of Education, National Center for Education 
Statistics (2007). 
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TABLE C2 

Descriptive statistics for multivariate analysis for math in Kentucky, 2002/03-2005/06 



Grade and variable 


Number of 
schools 


Mean 


Standard 

deviation 


Grade 5 


Change in percentage testing advanced, 2002/03-2005/06 


579 


3.28 


3.20 


Change in percentage passing, 2002/03-2005/06 


579 


6.48 


4.99 


Percentage testing proficient, 2001/02 


579 


29.84 


11.37 


Percentage of students eligible for free or reduced-price lunch 


579 


59.01 


23.10 


Change in percentage of students eligible for free or reduced-price lunch 


579 


3.19 


13.45 


Percentage of racial/ethnic minority students 


579 


13.17 


17.22 


Change in percentage of racial/ethnic minority students 


579 


1.43 


3.93 


City 


579 


0.21 


0.41 


Suburb 


579 


0.14 


0.34 


Town 


579 


0.17 


0.38 


Rural (comparison) 


579 


0.48 


0.50 


Grade 8 


Change in percentage testing advanced, 2002/03-2005/06 


194 


0.44 


1.36 


Change in passing, 2002/03-2005/06 


194 


1.21 


2.73 


Percentage testing proficient, 2001/02 


194 


21.09 


8.12 


Percentage of students eligible for free or reduced-price lunch 


194 


47.51 


19.40 


Change in percentage of students eligible for free or reduced-price lunch 


194 


4.60 


11.69 


Percentage of racial/ethnic minority students 


194 


11.22 


14.13 


Change in percentage of racial/ethnic minority students 


194 


1.57 


3.98 


City 


194 


0.19 


0.39 


Suburb 


194 


0.12 


0.32 


Town 


194 


0.29 


0.46 


Rural (comparison) 


194 


0.40 


0.49 


Grade 11 


Change in percentage testing advanced, 2002/03-2005/06 


194 


1.32 


1.51 


Change in passing, 2002/03-2005/06 


194 


1.82 


2.52 


Percentage testing proficient, 2001/02 


194 


20.62 


7.18 


Percentage of students eligible for free or reduced-price lunch 


194 


37.89 


18.01 


Change in percentage of students eligible for free or reduced-price lunch 


194 


4.75 


10.44 


Percentage of racial/ethnic minority students 


194 


9.43 


12.40 


Change in percentage of racial/ethnic minority students 


194 


1.40 


2.94 


City 


194 


0.13 


0.34 


Suburb 


194 


0.09 


0.29 


Town 


194 


0.29 


0.45 


Rural (comparison) 


194 


0.48 


0.50 



Source: Authors' calculations based on data from Kentucky Department of Education (2007); U.S. Department of Education, National Center for Education 
Statistics (2007). 
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TABLE C3 

Ordinary least squares multivariate regressions for reading in Kentucky, 2002/03-2005/06 





Model la 


Model 1b 


Model 1c 


Grade and variable 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Grade 4 


Constant 


0.82*** 


0.14 


-1.06* 


0.54 


-0.10 


0.91 


Change in percentage passing, 2002/03-2005/06 


0.30*** 


0.03 


0.31*** 


0.03 


0.31*** 


0.03 


Percentage testing proficient, 2001/02 






0.03*** 


0.01 


0.03** 


0.01 


Percentage of students eligible for free or 
reduced-price lunch 










-0.01 


0.01 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.02 


0.01 


Percentage of racial/ethnic minority students 










0.00 


0.01 


Change in percentage of racial/ethnic minority 
students 










-0.01 


0.03 


City 










0.16 


0.44 


Suburb 










-0.17 


0.41 


Town 










-0.18 


0.34 


Rural (comparison) 


R 2 


0.160 




0.179 




0.185 




Number of schools 


591 




591 




591 




Grade 7 


Constant 


-0.44*** 


0.10 


-0.64 


0.55 


-1.83 


1.12 


Change in percentage passing, 2002/03-2005/06 


0.30*** 


0.03 


0.30*** 


0.03 


0.30*** 


0.03 


Percentage testing proficient, 2001/02 






0.00 


0.01 


0.02 


0.02 


Percentage of students eligible for free or 
reduced-price lunch 










0.01 


0.01 


Change in percentage of students eligible for 
free or reduced-price lunch 










0.01 


0.01 


Percentage of racial/ethnic minority students 










0.01 


0.01 


Change in percentage of racial/ethnic minority 
students 










-0.01 


0.03 


City 










-0.21 


0.43 


Suburb 










-0.17 


0.37 


Town 










0.19 


0.26 


Rural (comparison) 


R 2 


0.327 




0.327 




0.338 




Number of schools 


194 




194 




194 
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TABLE C3 (CONTINUED) 

Ordinary least squares multivariate regressions for reading in Kentucky, 2002/03-2005/06 





Model la 


Model 1b 


Model 1c 


Grade and variable 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Grade 10 


Constant 


0.82*** 


0.17 


-1.00** 


0.33 


0.01 


0.68 


Change in percentage passing, 2002/03-2005/06 


0.52*** 


0.04 


0.53*** 


0.04 


0.53*** 


0.04 


Percentage testing proficient, 2001/02 






0.09*** 


0.01 


0.07*** 


0.02 


Percentage of students eligible for free or 
reduced-price lunch 










-0.01 


0.01 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.02 


0.01 


Percentage of racial/ethnic minority students 










-0.01 


0.01 


Change in percentage of racial/ethnic minority 
students 










-0.01 


0.04 


City 










0.34 


0.43 


Suburb 










-0.08 


0.40 


Town 










-0.16 


0.25 


Rural (comparison) 


R 2 


0.453 




0.545 




0.557 




Number of schools 


194 




194 




194 





* Significant at p < .05;** significant at p < .01;*** significant at p < .001. 

Source: Authors' calculations based on data from Kentucky Department of Education (2007); U.S. Department of Education, National Center for Education 
Statistics (2007). 
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TABLE C4 

Ordinary least squares multivariate regressions for math in Kentucky, 2002/03-2005/06 





Model la 


Model 1b 


Model 1c 


Grade and variable 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Grade 5 


Constant 


0.96*** 


(0.18) 


-1.27*** 


(0.35) 


0.66 


(0.63) 


Change in percentage passing, 2002/03-2005/06 


0.36*** 


(0.02) 


0.38*** 


(0.02) 


0.39*** 


(0.02) 


Percentage testing proficient, 2001/02 






0.07*** 


(0.01) 


0.05*** 


(0.01) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.02*** 


(0.01) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.01 


(0.01) 


Percentage of racial/ethnic minority students 










-0.00 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










0.01 


(0.03) 


City 










0.18 


(0.39) 


Suburb 










-0.34 


(0.36) 


Town 










-0.59 


(0.31) 


Rural (comparison) 


R 2 


0.313 




0.371 




0.391 




Number of schools 


591 




591 




591 




Grade 8 


Constant 


0.04 


(0.08) 


0.05 


(0.21) 


0.77 


(0.57) 


Change in percentage passing, 2002/03-2005/06 


0.33*** 


(0.03) 


0.33*** 


(0.03) 


0.32*** 


(0.03) 


Percentage testing proficient, 2001/02 






-0.00 


(0.01) 


-0.01 


(0.01) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.01 


(0.01) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.00 


(0.01) 


Percentage of racial/ethnic minority students 










0.01 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










-0.01 


(0.02) 


City 










-0.61 


(0.30) 


Suburb 










-0.22 


(0.26) 


Town 










-0.36 


(0.18) 


Rural (comparison) 


R 2 


0.442 




0.442 




0.466 




Number of schools 


194 




194 




194 
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TABLE C4 (CONTINUED) 

Ordinary least squares multivariate regressions for math in Kentucky, 2002/03-2005/06 





Model la 


Model 1b 


Model 1c 


Grade and variable 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Grade 11 


Constant 


0.64*** 


(0.10) 


-0.17 


(0.26) 


0.06 


(0.57) 


Change in percentage passing, 2002/03-2005/06 


0.38*** 


(0.03) 


0.39*** 


(0.03) 


0.39*** 


(0.03) 


Percentage testing proficient, 2001/02 






0.04*** 


(0.01) 


0.04* 


(0.02) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.00 


(0.01) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.00 


(0.01) 


Percentage of racial/ethnic minority students 










0.00 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










-0.02 


(0.03) 


City 










-0.03 


(0.35) 


Suburb 










-0.25 


(0.32) 


Town 










0.09 


(0.21) 


Rural (comparison) 


R 2 


0.401 




0.434 




0.442 




Number of schools 


194 




194 




194 





* Significant at p < .05;** significant at p < .01;*** significant at p < .001. 

Source: Authors' calculations based on data from Kentucky Department of Education (2007); U.S. Department of Education, National Center for Education 
Statistics (2007). 
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TABLE C5 

Descriptive statistics for multivariate analysis for reading in Virginia, 2002/03-2004/05 



Grade and variable 


Number of 
schools 


Mean 


Standard 

deviation 


Grade 3 


Change in percentage testing advanced, 2001/02-2004/05 


708 


-0.22 


4.31 


Change in percentage passing, 2002/03-2004/05 


708 


2.12 


5.15 


Percentage testing proficient, 2001/02 


708 


55.61 


8.90 


Percentage of students eligible for free or reduced-price lunch 


708 


32.19 


22.26 


Change in percentage of students eligible for free or reduced-price lunch 


708 


2.97 


6.99 


Percentage of racial/ethnic minority students 


708 


37.62 


26.26 


Change in percentage of racial/ethnic minority students 


708 


5.72 


7.20 


City 


708 


0.25 


0.43 


Suburb 


708 


0.40 


0.49 


Town 


708 


0.05 


0.21 


Rural 


708 


0.19 


0.39 


Grade 5 


Change in percentage testing advanced, 2001/02-2004/05 


734 


4.98 


4.24 


Change in percentage passing, 2002/03-2004/05 


734 


1.38 


4.10 


Percentage testing proficient, 2001/02 


734 


60.74 


8.42 


Percentage of students eligible for free or reduced-price lunch 


734 


32.35 


22.15 


Change in percentage of students eligible for free or reduced-price lunch 


734 


2.98 


7.17 


Percentage of racial/ethnic minority students 


734 


37.49 


26.52 


Change in percentage of racial/ethnic minority students 


734 


5.58 


7.15 


City 


734 


0.24 


0.43 


Suburb 


734 


0.40 


0.49 


Town 


734 


0.05 


0.21 


Rural 


734 


0.20 


0.40 


Grade 8 


Change in percentage testing advanced, 2002/03-2004/05 


280 


3.60 


3.70 


Change in percentage passing, 2002/03-2004/05 


280 


3.07 


3.33 


Percentage testing proficient, 2001/02 


280 


51.32 


6.62 


Percentage of students eligible for free or reduced-price lunch 


280 


31.44 


19.68 


Change in percentage of students eligible for free or reduced-price lunch 


280 


4.57 


9.69 


Percentage of racial/ethnic minority students 


280 


36.02 


25.67 


Change in percentage of racial/ethnic minority students 


280 


4.73 


5.47 


City 


280 


0.25 


0.44 


Suburb 


280 


0.31 


0.46 


Town 


280 


0.06 


0.25 


Rural 


280 


0.25 


0.43 
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TABLE C5 (CONTINUED) 

Descriptive statistics for multivariate analysis for reading in Virginia, 2002/03-2004/05 



Grade and variable 


Number of 
schools 


Mean 


Standard 

deviation 


FHigh school reading (end of course exam) 


Change in percentage testing advanced, 2002/03-2004/05 


248 


-2.17 


4.11 


Change in percentage passing, 2002/03-2004/05 


248 


-1.95 


3.09 


Percentage testing proficient, 2001/02 


248 


59.34 


7.92 


Percentage of students eligible for free or reduced-price lunch 


248 


23.57 


15.95 


Change in percentage of students eligible for free or reduced-price lunch 


248 


5.10 


9.08 


Percentage of racial/ethnic minority students 


248 


31.41 


24.51 


Change in percentage of racial/ethnic minority students 


248 


3.39 


4.27 


City 


248 


0.17 


0.38 


Suburb 


248 


0.23 


0.42 


Town 


248 


0.09 


0.30 


Rural 


248 


0.36 


0.47 



Source: Authors' calculations based on data from Virginia Department of Education (2007); U.S. Department of Education, National Center for Education 
Statistics (2007). 
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TABLE C6 

Descriptive statistics for multivariate analysis for math in Virginia, 2002/03-2004/05 



Grade and variable 


Number of 
schools 


Mean 


Standard 

deviation 


Grade 3 


Change in percentage testing advanced, 2002/03-2004/05 


716 


2.16 


6.13 


Change in percentage passing, 2002/03-2004/05 


716 


2.17 


4.09 


Percentage testing proficient, 2001/02 


716 


40.48 


8.74 


Percentage of students eligible for free or reduced-price lunch 


716 


32.43 


22.30 


Change in percentage of students eligible for free or reduced-price lunch 


716 


3.02 


6.99 


Percentage of racial/ethnic minority students 


716 


38.07 


26.61 


Change in percentage of racial/ethnic minority students 


716 


5.66 


7.21 


City 


716 


0.25 


0.44 


Suburb 


716 


0.40 


0.49 


Town 


716 


0.04 


0.21 


Rural 


716 


0.19 


0.39 


Grade 5 


Change in percentage testing advanced, 2002/03-2004/05 


736 


3.87 


4.75 


Change in percentage passing, 2002/03-2004/05 


736 


3.21 


5.51 


Percentage testing proficient, 2001/02 


736 


55.59 


10.59 


Percentage of students eligible for free or reduced-price lunch 


736 


32.49 


22.19 


Change in percentage of students eligible for free or reduced-price lunch 


736 


2.99 


7.17 


Percentage of racial/ethnic minority students 


736 


37.42 


26.61 


Change in percentage of racial/ethnic minority students 


736 


5.60 


7.17 


City 


736 


0.24 


0.43 


Suburb 


736 


0.40 


0.49 


Town 


736 


0.05 


0.21 


Rural 


736 


0.20 


0.40 


Grade 8 


Change in percentage testing advanced, 2002/03-2004/05 


318 


1.37 


3.29 


Change in percentage passing, 2002/03-2004/05 


318 


2.57 


4.26 


Percentage testing proficient, 2001/02 


318 


52.86 


9.91 


Percentage of students eligible for free or reduced-price lunch 


318 


31.82 


19.43 


Change in percentage of students eligible for free or reduced-price lunch 


318 


4.71 


9.37 


Percentage of racial/ethnic minority students 


318 


33.70 


25.79 


Change in percentage of racial/ethnic minority students 


318 


4.42 


5.40 


City 


318 


0.23 


0.42 


Suburb 


318 


0.29 


0.45 


Town 


318 


0.08 


0.26 


Rural 


318 


0.28 


0.45 
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TABLE C6 (CONTINUED) 

Descriptive statistics for multivariate analysis for math in Virginia, 2002/03-2004/05 



Grade and variable 


Number of 
schools 


Mean 


Standard 

deviation 


FHigh school Algebra 2 (end of course exam) 


Change in percentage testing advanced, 2002/03-2004/05 


210 


2.11 


3.32 


Change in percentage passing, 2002/03-2004/05 


210 


3.71 


4.71 


Percentage testing proficient, 2001/02 


210 


54.18 


9.96 


Percentage of students eligible for free or reduced-price lunch 


210 


20.98 


13.25 


Change in percentage of students eligible for free or reduced-price lunch 


210 


5.25 


6.74 


Percentage of racial/ethnic minority students 


210 


31.71 


23.16 


Change in percentage of racial/ethnic minority students 


210 


3.87 


4.33 


City 


210 


0.20 


0.40 


Suburb 


210 


0.26 


0.44 


Town 


210 


0.10 


0.30 


Rural 


210 


0.34 


0.45 



Source: Authors' calculations based on data from Virginia Department of Education (2007); U.S. Department of Education, National Center for Education 
Statistics (2007). 
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TABLE C7 

Ordinary least squares multivariate regressions for reading in Virginia, 2002/03-2004/05 





Model 2a 


Model 2b 


Model 2c 


Grade and variable 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Grade 3 


Constant 


-1.09*** 


(0.15) 


-2.89** 


(0.95) 


-3.55*** 


(1.02) 


Change in percentage passing 2002/03-2004/05 


0.41*** 


(0.03) 


0.42*** 


(0.03) 


0.42*** 


(0.03) 


Percentage testing proficient, 2001/02 






0.03 


(0.02) 


0.04* 


(0.02) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.02 


(0.01) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.02 




Percentage of racial/ethnic minority students 










0.03** 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










-0.00 




City 










0.38 


(0.38) 


Suburb 










-0.87* 


(0.32) 


Town 










0.50 


(0.63) 


Rural (comparison) 


R 2 


0.237 




0.240 




0.258 




Number of schools 


708 




708 




708 




Grade 5 


Constant 


4.60*** 


(0.16) 


1.90 


(1.13) 


6.81*** 


(1.19) 


Change in percentage passing 2002/03-2004/05 


0.28*** 


(0.04) 


0.30*** 


(0.04) 


0.35*** 


(0.04) 


Percentage testing proficient, 2001/02 






0.04* 


(0.02) 


-0.00 


(0.02) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.08*** 


(0.01) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.06** 


(0.02) 


Percentage of racial/ethnic minority students 










0.01 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










0.02 


(0.02) 


City 










0.92* 


(0.45) 


Suburb 










-0.04 


(0.40) 


Town 










0.86 


(0.73) 


Rural (comparison) 


R 2 


0.073 




0.078 




0.184 




Number of schools 


734 




734 




734 
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TABLE C7 (CONTINUED) 

Ordinary least squares multivariate regressions for reading in Virginia, 2002/03-2004/05 





Model 2a 


Model 2b 


Model 2c 


Grade and variable 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Grade 8 


Constant 


2.70*** 


(0.29) 


-0.487 


(1.76) 


5.79** 


(1.87) 


Change in percentage passing 2002/03-2004/05 


0.31*** 


(0.06) 


0.33*** 


(0.07) 


0.43*** 


(0.06) 


Percentage testing proficient, 2001/02 






0.06 


(0.03) 


-0.01 


(0.03) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.08*** 


(0.02) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.07** 


(0.02) 


Percentage of racial/ethnic minority students 










-0.01 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










-0.10*** 


(0.04) 


City 










2.35*** 


(0.62) 


Suburb 










-0.02 


(0.55) 


Town 










0.18 


(0.87) 


Rural (comparison) 


R 2 


0.072 




0.080 




0.223 




Number of schools 


280 




280 




280 




FHigh school reading (end of course exam) 


Constant 


-1.18*** 


(0.28) 


-5.80*** 


(1.79) 


-3.829* 


(1.91) 


Change in percentage passing 2002/03-2004/05 


0.54*** 


(0.08) 


0.56*** 


(0.08) 


0.58*** 


(0.08) 


Percentage testing proficient, 2001/02 






0.08** 


(0.03) 


0.06 


(0.03) 


Percentage of students eligible for free or 
reduced-price lunch 










0.01 


(0.02) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.07* 


(0.03) 


Percentage of racial/ethnic minority students 










-0.01 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










-0.10 


(0.06) 


City 










0.89 


(3.76) 


Suburb 










-0.28 


(0.69) 


Town 










-0.03 


(0.80) 


Rural (comparison) 


R 2 


0.165 




0.189 




0.239 




Number of schools 


248 




248 




248 





* Significant at p < .05;** significant at p < .01;*** significant at p < .001. 

Source: Authors' calculations based on data from Virginia Department of Education (2007); U.S. Department of Education, National Center for Education 
Statistics (2007). 
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TABLE C8 

Ordinary least squares multivariate regression for math in Virginia, 2002/03-2004/05 





Model 2a 


Model 2b 


Model 2c 


Grade and variable 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Grade 3 


Constant 


0.09 


(0.20) 


1.27 


(0.84) 


2.05 


(0.91) 


Change in percentage passing, 2002/03-2004/05 


0.95*** 


(0.04) 


0.96*** 


(0.04) 


0.98*** 


(0.05) 


Percentage testing proficient, 2001/02 






-0.03 


(0.02) 


-0.01 


(0.02) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.03 


(0.01) 


Change in percentage of students eligible for 
free or reduced-price lunch 










0.05 


(0.03) 


Percentage of racial/ethnic minority students 










0.00 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










-0.06* 


(0.03) 


City 










-0.82 


(0.56) 


Suburb 










-0.63 


(0.50) 


Town 










-1.52 


(0.93) 


Rural (comparison) 


R 2 


0.401 




0.402 




0.418 




Number of schools 


716 




716 




716 




Grade 5 


Constant 


2.43*** 


(0.17) 


-0.10 


(0.84) 


2.86** 


(0.99) 


Change in percentage passing, 2002/03-2004/05 


0.45*** 


(0.03) 


0.47*** 


(0.03) 


0.50*** 


(0.03) 


Percentage testing proficient, 2001/02 






0.04** 


(0.01) 


0.01 


(0.02) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.02 


(0.01) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.00 


(0.02) 


Percentage of racial/ethnic minority students 










-0.03*** 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










-0.03 


(0.03) 


City 










1.25** 


(0.46) 


Suburb 










0.71 


(0.41) 


Town 










0.41 


(0.75) 


Rural (comparison) 


R 2 


0.274 




0.283 




0.323 




Number of schools 


736 




736 




736 





(CONTINUED) 




50 



PERCENTAGE OF STUDENTS PASSING & PERCENTAGE TESTING ADVANCED ON KENTUCKY & VIRGINIA ASSESSMENTS 



TABLE C8 (CONTINUED) 

Ordinary least squares multivariate regression for math in Virginia, 2002/03-2004/05 





Model 2a 


Model 2b 


Model 2c 


Grade and variable 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Coefficient 


Standard 

error 


Grade 8 


Constant 


0.60** 


(0.20) 


-2.48* 


(0.99) 


-0.548 


(1.16) 


Change in percentage passing, 2002/03-2004/05 


0.30*** 


(0.04) 


0.34*** 


(0.04) 


0.36*** 


(0.04) 


Percentage testing proficient, 2001/02 






0.06** 


(0.02) 


0.03 


(0.02) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.04*** 


(0.01) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.04* 


(0.02) 


Percentage of racial/ethnic minority students 










0.03*** 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










-0.00 


(0.03) 


City 










-1.27* 


(0.53) 


Suburb 










0.65 


(0.46) 


Town 










-0.56 


(0.65) 


Rural (comparison) 


R 2 


0.150 




0.176 




0.265 




Number of schools 


318 




318 




318 




FHigh school Algebra 2 (end of course exam) 


Constant 


0.93*** 


(0.26) 


-2.51* 


(1.17) 


-2.83 


(1.24) 


Change in percentage passing, 2002/03-2004/05 


0.32*** 


(0.04) 


0.34*** 


(0.04) 


0.35*** 


(0.04) 


Percentage testing proficient, 2001/02 






0.06** 


(0.02) 


0.07** 


(0.02) 


Percentage of students eligible for free or 
reduced-price lunch 










-0.01 


(0.02) 


Change in percentage of students eligible for 
free or reduced-price lunch 










-0.03 


(0.03) 


Percentage of racial/ethnic minority students 










0.00 


(0.01) 


Change in percentage of racial/ethnic minority 
students 










0.05 


(0.05) 


City 










-0.78 


(0.64) 


Suburb 










0.63 


(0.56) 


Town 










1.99** 


(0.72) 


Rural (comparison) 


R 2 


0.199 




0.233 




0.291 




Number of schools 


210 




210 




210 





* Significant at p < .05;** significant at p < .01;*** significant at p < .001. 

Source: Authors' calculations based on data from Virginia Department of Education (2007); U.S. Department of Education, National Center for Education 
Statistics (2007). 
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